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ABSTRACT 


Asymmetric  threats  pose  increasing  challenges  to  the  United  States  Navy  in  littoral 
environments.  To  address  the  Navy’s  need  for  a  new  platform  to  serve  in  this  area,  the 
Littoral  Combat  Ship  (LCS)  was  designed  and  put  into  service.  What  still  has  yet  to  be 
determined  is  what  surface-to-surface  capability  the  LCS  will  have  as  well  as  what  air-to- 
surface  capability  the  LCS  helicopter/unmanned  aerial  vehicle  (UAV)  will  have. 

This  study  uses  freely  available  data  to  build  a  simulation  utilizing  an  agent-based 
modeling  platform  known  as  MANA.  The  simulation  is  exercised  over  a  broad  range  of 
different  weapon  systems  types  with  their  capabilities  ranged  across  the  spectrum  of 
possibilities  based  on  their  effectiveness  as  well  as  potential  difficulties  in  targeting  small 
boat  threats.  Using  linear  regression  and  partition  trees,  an  analysis  is  performed  on  the 
resulting  dataset  to  address  the  research  question. 

The  results  show  that  the  NLOS  system  is  the  best  surface-to-surface  missile 
system  for  the  LCS  as  long  as  the  expected  rate  of  fire  is  obtained.  The  best  air-to- 
surface  missile  system  is  either  APKWS  or  LOGIR,  depending  on  which  can  obtain  a 
rate  of  fire  of  one  missile  every  nine  seconds  or  faster.  Lastly,  the  rate  of  fire  has  been 
shown  to  be  the  most  important  factor  in  determining  the  effectiveness  of  the  different 
missiles. 
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I.  INTRODUCTION 


A.  OVERVIEW 

With  the  ending  of  the  Cold  War,  the  United  States  Navy  is  seeing  its  mission 
change.  Gone  are  the  days  of  the  threat  of  an  engagement  with  a  large  blue  water  navy  as 
represented  by  the  Soviet  Navy.  Today’s  challenge  is  from  smaller  nations  that  employ 
diesel  submarines,  fast  missile  armed  patrol  boats,  and  shore-based  cruise  missiles  in  an 
attempt  to  deny  access  to  U.S.  forces.  Another  new  threat  is  the  development  of  more 
capable  unmanned  remote  control  aircraft,  boats,  and  submersibles.  The  Navy  realized 
that  to  counter  these  threats  would  require  a  ship  capable  of  operating  in  the  littoral 
environment  and  able  to  incorporate  these  newer  unmanned  technologies.  This  resulted 
in  the  design  of  the  modular,  focused,  mission  platform  known  as  the  Littoral  Combat 
Ship  (LCS);  see  Figure  1  and  Figure  2. 


Figure  1.  LCS-1,  First  ship  of  the  Lockheed  Martin  Design  (From  Jane’s,  2010) 
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Figure  2.  LCS-2,  First  ship  of  the  General  Dynamics  Design  (From  Jane’s,  2010) 

The  LCS  has  several  different  mission  modules  that  give  it  the  ability  to  be 
utilized  in  a  specific  warfare  area.  These  modules  are  the  Anti-Submarine  Warfare 
(ASW),  Mine  Warfare  (MIW),  and  Surface  Warfare  (SUW). 

B.  RESEARCH  QUESTIONS 

This  thesis  looks  at  a  LCS  equipped  with  the  SUW  module  and  with  an  embarked 
aircraft  and  attempts  to  see  how  the  LCS  handles  itself  in  an  over-the-horizon  surface 
engagement  against  missile-armed  patrol  boats.  The  following  questions  guide  this 
research: 

•  What  surface-to-surface  missile  should  LCS  equip? 

•  What  type  of  air-to-surface  missiles  should  the  aircraft  be  equipped  with 
to  be  the  most  effective? 

•  What  factor  is  more  important  in  determining  the  type  of  missile  to  select? 

This  thesis  uses  simulation  models,  data  analysis,  and  other  analytical  tools  to 
investigate  these  questions  and  detennine  which  combination  of  missiles  and  aircraft 
provides  for  the  best  overall  SUW  package.  The  simulation  model  utilizes  a  given  littoral 
region  and  threats  that  may  exist  in  that  area. 
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c. 


BENEFITS  OF  THE  STUDY 


This  thesis  provides  the  U.S.  Navy  analytical  support  for  the  customization  of  the 
SUW  module  package  for  the  LCS.  Additionally,  this  study  provides  insight  into  the 
comparative  capabilities  of  different  air-to-surface  missiles  as  well  as  an  insight  into 
which  missiles  will  offer  more  ‘bang  for  the  buck.’ 

D.  ORGANIZATION  OF  THE  DOCUMENT 

Chapter  II  consists  of  the  literature  review,  exploring  the  background  and 
capabilities  of  the  platfonns  and  missiles  used  in  this  thesis.  Chapter  III  describes  the 
modeling  tool  used  in  this  thesis  and  the  reasoning  for  selecting  it.  Chapter  IV  describes 
the  design  of  the  experiment  with  respect  to  behaviors  of  simulated  entities  and  creation 
of  the  scenarios.  Chapter  V  contains  analysis  and  conclusions  as  well  as 
recommendations  for  future  work. 
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II.  LITERATURE  REVIEW 


A.  INTRODUCTION 

This  chapter  covers  the  current  threats  and  deficiencies  within  the  U.S.  Navy  that 
have  necessitated  the  launching  of  the  LCS  class. 

Operating  in  the  littoral  presents  a  complex  collection  of  challenges.  As 
...From  the  Sea  put  it,  the  “mastery  of  the  littoral  should  not  be 
presumed.”  ...From  the  Sea  recognized  that  “Some  littoral  threats- 
specifically  mines,  sea-skimming  cruise  missiles,  and  tactical  ballistic 
missiles-tax  the  capabilities  of  our  current  systems  and  force  structure.”  In 
the  past  decade,  swarming  small  boats  (armed  with  short  range  missiles  or 
a  payload  of  explosives)  and  diesel  submarines  have  also  been  cited  by  the 
Navy  as  obstructing  U.S.  access  to  the  littoral.  These  systems  enable  even 
relatively  unsophisticated  adversaries  to  adopt  a  strategy  of  anti-access 
and  area  denial  (A2/AD),  whereby  the  defender  seeks  to  prevent  the 
attacker  from  bringing  strike  power  to  bear  with  a  layered,  but  not 
symmetric  force-on-force,  defense  of  the  approaches  (Long/Johnson 
2007). 

B.  LCS  PROGRAM  BACKGROUND 
1.  History 

Faced  with  new  threats  in  the  littoral  region,  pictured  below  in  Figure  3,  the  Navy 
strategy  initially  focused  on  avoiding  the  littoral  and  projecting  power  over  the  littorals 
using  gunfire,  missiles,  and  air  power  delivered  by  platfonns  already  in  service  in  the 
fleet.  This  would  minimize  exposure  to  mines  and  land-based  anti-ship  missiles.  This 
strategy  of  avoiding  the  littoral  resulted  in  the  DD-21  Land  Attack  Destroyer  program 
which  the  primary  mission  foreseen  to  be  to  support  ground  forces.  Over  time,  however, 
elements  within  the  Navy  begin  to  recognize  the  difficulty  of  supporting  ground  forces 
without  a  more  persistent  presence  within  the  littoral  environment  and  proposed  filling 
this  gap  with  new,  small  surface  combatants.  Recognition  of  this  difficulty  was  furthered 
by  a  1997  General  Accounting  Office  (GAO)  report  which  indicated  that  the  Navy’s 
shipbuilding  program  was  facing  a  significant  fiscal  challenge  and  that  if  the  Navy  was  to 
meet  its  force  structure  goal  then  it  would  have  to  either  buy  less  expensive  ships  or 
spend  more.  The  suggestion  from  GAO  to  the  Navy  was  to  consider  cheaper  ships  that 
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would  be  tailored  to  perform  only  one  or  two  missions.  This  cause  was  furthered  by  the 
Chief  of  Naval  Operations’  Strategic  Studies  Group  (SSG),  which  recommended  that  the 
two  most  important  attributes  for  the  future  fleet  were  for  distributed  combat  power, 
achievable  through  use  of  unmanned  vehicles  and  offboard  sensors,  and  modularity  to 
provide  mission  flexibility  (Long/Johnson  2007).  What  finally  caused  the  Navy  to  act  on 
this  need  was  the  release  of  a  report  by  the  GAO  in  2001  citing  the  need  for  improved 
war-fighting  capabilities  in  the  littoral  region  (GAO  2001).  The  Navy  launched  a  study 
in  2001  to  find  the  ship  that  would  address  all  of  its  needs  for  a  littoral  capable  warship. 
The  Navy  decided  upon  the  Littoral  Combat  Ship,  a  smaller  and  less  expensive  ship, 
when  compared  to  destroyers  and  cruisers,  that  depending  on  which  module  it  is 
equipped  with,  could  handle  the  mine,  diesel  submarine,  and  small  boat  threats  in  the 
littoral  environment.  The  LCS  design  does  not,  however,  have  an  anti-air  capability 
outside  of  limited  self-defense. 


“The  Littoral  Niche 
er  Challenge” 


Figure  3.  The  overlapping  threat  environment  in  the  littoral  region  (From  Naval  Warfare 

Development  Command,  2007) 
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2. 


What  Is  the  Littoral  Combat  Ship? 


a.  Overview 

The  LCS  is  planned  to  fill  the  need  for  a  combatant  in  the  littoral  regions 
of  the  world  and  to  replace  several  classes  of  ships  that  are  being  retired,  most  notably 
frigates  and  minesweepers.  Flexibility  is  the  defining  characteristic  of  LCS  as  it  is 
expected  to  operate  in  a  littoral  region  in  the  SUW,  MIW,  and  ASW  missions. 

The  LCS  is  a  relatively  inexpensive  Navy  surface  combatant  that  is  to  be 
equipped  with  modular  “plug-and-fight”  mission  packages,  including 
unmanned  vehicles  (UVs).  Rather  than  being  a  multimission  ship  like  the 
Navy’s  larger  surface  combatants,  the  LCS  is  to  be  a  focused-mission  ship 
equipped  to  perfonn  one  primary  mission  at  any  one  time.  The  ship’s 
mission  orientation  can  be  changed  by  changing  its  mission  packages. 

The  basic  version  of  the  LCS,  without  any  mission  packages,  is  referred  to 
as  the  LCS  sea  frame.  (Navy  Littoral  Combat  Ship  (LCS)  Program: 
Background,  Issues,  and  Options  for  Congress,  April  8,  2010) 

b.  Seaframe 

As  the  core  of  LCS,  the  seaframe  provides  for  basic  self-defense  capability 
through  organic  sensors,  weapons,  and  speed.  There  are  two  seaframe  designs  that  are 
being  considered  with  one  of  each  already  in  service.  There  are  differences  between  the 
two  seaframes  but  those  are  not  the  focus  of  this  thesis.  The  two  seaframes  are  capable  of 
40+  knots  and  are  similarly  equipped  regarding  organic  sensors  and  weaponry.  The  only 
organic  capabilities  that  this  thesis  looks  at  are  the  speed  of  40+  knots  and  the  radar 
system.  The  radar  is  the  European  Aeronautic  Defense  and  Space  Company  (EADS) 
Three-Dimensional  Track  Reporting  System  (TRS-3D),  which  is  a  naval  multimode 
three-dimensional  air/surface  search  radar  with  periscope  detection  capability. 

c.  Capabilities 

The  focus  of  this  thesis  is  on  the  SUW  module  equipped  LCS.  The  SUW 
module  is  designed  to  detect  and  engage  multiple  surface  contacts  in  a  littoral 
environment.  It  strengthens  the  core  seaframe  capability  by  adding  an  air-to-surface 
missile  armed  aircraft  and  a  surface-to-surface  missile  capability.  According  to  the  Naval 
Sea  System  Command: 
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These  components  include  electro-optical/infrared  sensors  mounted  on  a 
vertical  takeoff  unmanned  air  vehicle  to  provide  over-the-horizon 
detection;  30mm  guns  to  kill  close-in  targets;  four  non-line-of-sight 
launching  system  (NLOS-LS)  container  launch  units  or  “missile-in-a-box” 
systems,  with  each  system  containing  15  offensive  missiles;  and  the  MH- 
60R  armed  helicopter  for  surveillance  and  attack  missions(U.S.  Navy, 

2007). 

3.  Issues 

There  are  some  concerns  about  the  LCS  program.  Some  concerns  stem  from  the 
rising  costs  of  the  ships,  some  from  the  slow  development  of  the  required  technologies 
for  the  warfare  modules,  and  some  from  the  combat  survivability  of  the  seaframes. 

a.  Combat  Survivability 

There  are  concerns  about  the  survivability  of  the  LCS  in  a  combat 
situation  due  to  its  small  crew  size,  which  limits  damage  control  operations,  and  the 
limited  shock  hardened  design  of  both  seaframes. 

The  LCS  is  not  expected  to  be  survivable  in  a  hostile  combat  environment 
as  evidenced  by  the  limited  shock  hardened  design  and  results  of  full  scale 
testing  of  representative  hull  structures  completed  in  December  2006 
(O’Rourke  2010,  p.  17). 

b.  Development  of  Required  Technologies 

Of  the  19  critical  technologies  required  for  the  two  different  seaframe 
options,  only  15  can  be  said  to  be  fully  mature  and  of  the  25  critical  technologies  required 
for  the  MIW,  SUW,  and  ASW  modules  only  17  are  currently  mature  (GAO  March  2009). 
Not  only  has  the  development  been  behind  what  was  expected,  but  the  development  cost 
of  the  modules  and  the  seaframes  have  been  much  higher  than  expected. 

4.  Suitability 

Some  that  feel  that  the  LCS  is  not,  in  fact,  the  right  ship  for  the  littoral 
environment.  One  such  person  is  Milan  Vego,  who  in  an  article  in  the  Armed  Forces 
Journal  writes,  “The  best  weapon  to  counter  enemy  small  surface  combatants  is  a  force  of 
small  surface  combatants”  (Vego  2008).  He  goes  on  to  state  that  the  LCS  is  not  a  real 


8 


littoral  combat  vessel  but  is  an  ocean-going  vessel  due  to  its  draft  of  20  feet.  He  feels  this 
will  limit  the  maneuverability  of  the  LCS  in  the  confined  waters  of  a  littoral  environment 
and  that  no  matter  how  well  equipped  it  might  be  that  it  would  not  be  able  to  defend 
against  a  large  number  of  hostile  small  boats.  He  is  mistaken  in  his  talk  of  the  draft  of  the 
LCS.  The  Freedom  class  has  a  draft  of  13.5  feet  and  the  Independence  class  has  a  draft  of 
14.8  feet.  Martin  Murphy,  of  the  Center  for  Strategic  and  Budgetary  Assessment  came 
out  in  favor  of  the  LCS.  He  makes  the  point  that  the  LCS  with  its  high  speed  and  shallow 
draft  of  15  feet  is  a  kind  of  “light  cavalry”  for  the  Navy.  He  also  points  out  that  the  real 
value  of  the  LCS  comes  from  its  large  flight  deck  and  its  ability  to  operate  two  MH-60R 
helicopters  or  an  assortment  of  drones;  moreover,  a  MH-60R  is  faster  and  more 
maneuverable  than  any  small  boat  and  when  anned  with  Hellfire  Missiles  it  is  a  very 
lethal  aircraft  (Murphy  2010). 

C.  CONCEPT  OF  OPERATIONS 

Figure  4  is  an  excellent  illustration  of  how  the  U.S.  Navy  envisions  LCS  being 
used.  The  Navy  refers  to  this  as  a  distributed  force  in  the  littoral,  not  platform-centric 
(Global  Security  2003).  The  Navy  sees  Littoral  Combat  Ships,  each  with  appropriate 
network  of  off  board  sensors  and  systems,  being  able  to  operate  independently  or 
interdependently  as  part  of  a  littoral  operations  force  or  a  multi-mission  fleet  force.  The 
point  is  for  the  LCS  to  be  networked  with  its  autonomous  vehicles,  whether  for  SUW, 
MIW,  or  ASW  and  to  use  those  vehicles  to  allow  LCS  to  investigate  contacts  or  shipping 
without  exposing  the  ship  or  any  of  its  crew  to  harm.  Looking  at  it  from  the  SUW  point 
of  view,  these  networked  UAVs  or  MH-60R  will  be  able  to  detect,  identify,  classify  and 
track  a  threat  and  with  the  Non  Line-of-Sight  Launch  System  (NLOS-LS)  that  the  LCS  is 
equipped  with,  it  will  be  able  to  engage  surface  threats  as  far  out  as  40  kilometers.  This 
means  that  the  SUW  LCS  will  have  the  capability  to  handle  most  threats  outside  of  their 
engagement  envelope  thereby  limiting  the  loss  of  equipment  and  life  to  that  of  the  UAV 
or  the  MH-60R.  This  also  means  that  the  UAV  or  MH-60R  can  limit  their  risk  by 
remaining  just  close  enough  to  laser-designate  the  target  for  the  ship  without  getting 
within  the  range  of  any  hand-held  Surface-to-Air  Missiles  (SAMs)  that  might  be  onboard 
the  threat  boat. 
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Distributed  Off-Board  Systems 


SAT  or  UAV  Relay 


LCS  focused  missions  exploit 
power  of  distributed  systems 


Figure  4.  How  the  U.S.  Navy  envisions  LCS  utilizing  automated  vehicles  to  complete  its 
missions.  (From  Naval  Warfare  Development  Command,  2007) 

D.  ISSUES 

There  is  a  major  issue  with  the  current  plan  for  the  SUW  module  equipped  LCS 
and  that  is  the  development  of  the  NLOS-LS.  Originally  an  Army  program  it  was 
discovered  earlier  this  year  that  the  NLOS-LS’ s  Precision  Attack  Missile  (PAM)  is  not 
very  precise,  missing  four  out  of  six  targets  in  tests  conducted  in  at  White  Sands  Missile 
Range,  New  Mexico,  between  January  26,  2010  and  February  5,  2010. 

Test  missiles  failed  to  hit  a  moving  tank  20  kilometers  away,  a  moving 
infantry  vehicle  10  kilometers  away,  a  stationary  tank  30  kilometers  away, 
and  a  stationary  truck  35  kilometers  away.  It  missed  the  infantry  vehicle 
by  20  meters,  and  the  truck  by  25  kilometers.  (Brannen  February  22, 

2010) 

It  is  note-worthy  that  the  two  hits  came  when  the  missile  used  its  laser  designator 
instead  of  its  infrared  seeker,  so  there  is  still  potential.  The  results  of  the  test  were 
alarming  enough  that  earlier  this  year  the  Army  canceled  the  NLOS-LS  program  and  the 


10 


funding  for  continued  research  and  development  switched  to  the  Navy  (Brannen  May  12, 
2010).  Another  cause  for  concern  is  the  price  tag;  each  PAM  costs  an  astounding 
$466,000. 

1.  Options 

These  issues  with  the  NLOS-LS  program  have  some  in  the  Navy  wondering  if  the 
NLOS-LS  program  is  the  right  fit  for  the  LCS  SUW  module.  Other  options  are  available 
to  provide  the  LCS  with  an  over-the-horizon  anti-surface  capability. 

a.  Current 

The  RGM-84  Harpoon  missile  is  the  current  U.S.  Navy  surface-to-surface 
anti-ship  missile.  It  is  an  existing  technology  and  is  still  in  production.  The  main 
downside  is  that  where  the  talk  for  NLOS  was  for  putting  45  or  60  PAMs  on  the  LCS, 
with  Harpoons  it  would  probably  be  limited  to  eight  missiles. 

b.  Future 

The  Navy  is  looking  ahead  for  a  next-generation  ship-launched  surface-to- 
surface  missile  (Peterson  2010),  but  as  of  April  2010,  the  Navy  was  still  drafting  an 
initial  capabilities  document,  which  means  that  any  next-generation  program  is  still  quite 
a  few  years  down  the  road.  Some  of  the  programs  that  have  potential  are  the  Naval  Strike 
Missile  (NSM)  of  the  Royal  Norwegian  Navy,  the  Multi-Mission  Tomahawk  (MMT),  or 
a  more  advanced  Harpoon  missile  (Peterson  2010). 

E.  AIRBORNE  PLATFORMS 

This  section  provides  a  brief  overview  and  capabilities  description  of  the  aircraft 
used  in  the  scenarios  for  this  thesis. 

1,  MH-60R 

a.  Overview 

The  MH-60R  is  one  of  the  latest  versions  of  the  Sikorsky  S-70B  helicopter 
that  has  been  in  use  in  the  U.S.  Navy  for  years.  Much  like  the  LCS,  the  MH-60R  is  a 
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multi-mission  platfonn.  It  is  equipped  to  conduct  both  ASW  and  SUW  as  its  primary 
missions  and  is  able  to  employ  Link  16  to  further  incorporate  it  into  a  strike  group’s 
network.  The  avionics  have  been  designed  to  be  non-mission  specific  so  that  the  pilot 
can  shift  from  a  search  and  rescue  (SAR)  mission  to  a  SUW  mission  to  an  ASW  mission 
and  be  looking  at  the  same  cockpit. 

b.  Capabilities 

The  MH-60R  combines  the  capabilities  of  the  SH-60B  with  the  dipping 
sonar  of  the  SH-60F.  Most  of  these  capabilities  are  for  ASW  and  are  of  no  interest  to  this 
thesis.  The  capabilities  that  are  relevant  to  this  thesis  are  that  the  MH-60R  is  outfitted  to 
carry  and  use  eight  Hellfire  missiles  and  is  equipped  with  the  Telephonies  AN/APS-147 
search  radar  and  the  Raytheon  AN/AAS-44  FLIR/laser  ranger.  The  AN/APS-147  has  the 
ability  to  auto  detect  and  track  up  to  255  contacts  simultaneously. 

2.  Northrop  Grumman  MQ-8  Fire  Scout 

a.  Overview 

Based  on  a  modified  Schweizer  333  light  helicopter,  the  MQ-8B  has  a 
four-blade  rotor,  a  streamlined  fuselage  pod  in  place  of  the  cabin,  and  increased  fuel 
capacity.  Due  to  delays  in  the  LCS  program,  a  ship  of  the  FFG-7  class,  the  USS 
Mclnerney,  was  selected  to  conduct  a  technical  evaluation  of  the  Fire  Scout.  Between 
December  2008  and  November  2009  110  ship  take-offs  and  landings  were  conducted. 
During  this  time  period  the  Fire  Scout  completed  a  successful  deployment  in  the  US 
Southern  Command  (SOUTHCOM)  area  of  responsibility  (AOR)  (Jane’s  April  21, 
2010). 


b.  Capabilities 

The  Fire  Scout  is  capable  of  125  knots  in  level  flight  and  has  a  mission 
radius  of  1 10  nautical  miles  (NM).  With  a  maximum  payload  of  272  kg  (600  lb)  it  has  an 
endurance  of  three  hours.  It  is  also  equipped  with  FLIR,  a  laser  designator,  and  the 
General  Atomics  AN/APY-8  Lynx  Synthetic  Aperture  Radar  (SAR)/Moving  Target 
Indication  (MTI)  radar. 
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F. 


MISSILES 


This  section  provides  a  brief  introduction  to  the  different  missile  systems  and 
missiles  relevant  to  this  thesis. 

1.  Non  Line  of  Sight-Launch  System  (NLOS-LS) 

The  NLOS  began  as  an  Army  project  being  developed  by  Lockheed  Martin. 
Since  then  it  has  become  a  Joint  Army-Navy  project  and  is  now  just  a  Navy  project.  The 
plan  was  to  integrate  the  NLOS-LS  into  the  LCS  but  the  future  of  the  program  is 
currently  in  question  due  to  some  subpar  test  results  in  which  the  missile  missed  four  out 
of  six  targets  (Defense  Tech,  2010).  The  design  concept  is  for  the  target  to  be  detected, 
identified,  classified,  and  tracked  by  an  off-board  surveillance  platform  and  then  for  the 
LCS  to  set  aim  points  for  the  Precision  Attack  Missile  (PAM).  LCS  will  send  continuous 
target  updates  to  the  missile  until  it  is  about  eight  km  from  the  target  at  which  time  each 
PAM  begins  to  zero-in  on  its  specified  target  using  its  MTI  and  its  automatic  target 
acquisition  capabilities.  The  missiles  are  contained  in  Container  Launching  Units 
(CLUs);  each  CLU  is  a  self-contained  16  cell  launching  system  that  has  15  missiles  and  a 
Computer  and  Communications  System  (CCS).  The  plan  is  for  four  CLUs  to  be  carried 
in  each  LCS  SUW  module. 


Figure  5.  NLOS  launching  from  CLU  (From  Defense  Tech,  2010) 

The  PAM  is  subsonic  and  has  a  range  of  40  km.  The  PAM  has  three  seeker 


options  available:  it  can  use  its  IR  seeker  to  search  for  and  lock  onto  its  target,  it  can  use 
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its  laser-guided  seeker  to  search  for  reflected  laser  energy  from  a  third  party  designator, 
or  it  can  use  GPS  to  fly  to  a  specified  GPS  location  and  detonate  on  impact.  The  PAMs 
can  be  fired  at  five-second  intervals  and  each  is  equipped  with  a  13.2  kg  multi-mission 
warhead.  Current  estimates  put  the  cost  of  each  NLOS  missile  at  $466,000  apiece 
(Defense  Tech,  2010). 

2.  RGM-84  Harpoon 

The  Harpoon  is  the  most  widely  used,  western  made,  ship-launched,  anti-ship 
missile.  It  has  been  in  use  in  the  U.S.  Navy  since  1977.  Although  it  has  not  seen  much 
use  of  late  in  the  U.S.  Navy,  having  not  been  included  on  the  flight  II  DDGs;  the  U.S. 
continues  to  fund  research  into  upgrades  and  is  continuing  to  upgrade  its  current  stock  of 
Harpoon  Missiles. 

The  latest  version  of  Harpoon  is  sea-skimming  capable  with  an  active  radar  seeker 
and  datalink,  meaning  it  can  be  updated  while  in  flight.  It  carries  a  222  kg  warhead  and 
has  proven  flight  reliability,  which  has  increased  over  the  years,  has  can  be  seen  in  the 
Table  1. 


14 


Table  1. 


Harpoon  Flight  Reliability  (From  Jane’s,  Feb  04,  2010) 


Fiscal  Year 

Missiles  launched 

Successes 

Success  percentage 

1975-76 

98 

87 

88.77 

1977-78 

73 

68 

93.15 

1979-81 

114 

106 

92.98 

1982-89 

136 

134 

98.5 

Total 

421 

395 

93.35 

3.  AGM-114  Hellfire 

The  Hellfire  (an  acronym  for  Heliborne,  Laser,  Fire  and  Forget)  Modular  Missile 
System  was  designed  as  an  anti-armor  and  precision  attack  weapon  in  the  1970s  with  the 
requirement  that  it  could  attack  both  stationary  and  moving  targets.  The  Hellfire  has 
been  cleared  for  use  with  several  helicopter  and  UAV  platforms  in  the  U.S.  inventory, 
including  the  MH-60R,  the  MQ-1  Predator  UAV,  and  AH-64  Apache.  The  Air  Force  is 
even  looking  at  integrating  it  with  the  next  generation  of  AC- 130  gunships. 

The  missiles  can  be  carried  on  two  or  four-rail  launcher  assemblies.  The  variant 
this  thesis  is  looking  at  is  the  AGM-114M,  which  has  a  12.5  kg  warhead,  a  maximum 
range  of  nine  kilometers,  and  uses  semi-active  laser  guidance. 

4.  Low-Cost  Guided  Imaging  Rocket  (LOGIR) 

There  have  been  several  efforts  by  the  U.S.  Navy  to  design  and  build  an 
affordable  precision  guidance  system  for  the  standard  Hydra  70  rocket.  The  LOGIR  is 
unlike  most  other  guided  rocket  programs,  in  that  it  relies  on  infrared  guidance  rather 
than  semi-active  laser  homing  technique.  As  of  2007,  LOGIR  was  being  co-operatively 
developed  by  the  United  States  and  South  Korea. 

The  LOGIR  modification  is  designed  to  turn  an  existing  Hydra  70  rocket  or 
CRV7  rocket  into  a  guided  rocket  with  the  addition  of  a  new  guidance  and  control  system 
(GCS).  This  system  incorporates  an  imaging  infrared  seeker  for  terminal  homing  and  an 
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inertial  navigation  platfonn.  This  seeker  uses  an  uncooled  staring  focal  plane  array 
(FPA)  that  uses  imaging  matching  to  locate  and  identify  its  target.  The  target  is  then 
designated  using  the  FLIR  sensor  on  the  launching  helicopter  or  UAV  and  the  LOGIR 
then  uses  that  data  to  track  and  acquire  the  target  itself.  LOGIR  is  intended  to  be  used  as 
a  fire-and-forget  weapon,  allowing  for  a  greater  rate  of  fire  than  a  semi-active  laser 
homing  guided  rocket.  The  effective  range  is  expected  to  be  about  five  kilometers  and 
since  it  is  using  existing  Hydra  70  rockets,  the  warhead  should  be  the  standard  7.7  kg 
(Jane’s).  Since  this  program  will  be  using  the  existing  Hydra  70  rockets,  there  is  a 
possibility  that  the  Office  of  Naval  Research  (ONR)  will  attempt  to  utilize  the  existing 
Hydra  70  rocket  launchers. 

5.  Advanced  Precision  Kill  Weapon  System  (APKWS) 

The  APKWS  program  is  another  initiative  by  the  United  States  military  to 
develop  the  standard  Hydra  70  rocket  and  the  CRV7  rocket  into  low-cost  precision- 
guided  weapons.  This  program  was  initiated  by  the  U.S.  Army  and  was  taken  over  by  the 
Navy  in  2008  with  the  intention  to  equip  Marine  Corps  combat  helicopters.  APKWS  has 
a  range  from  between  0.91  miles  to  3.1  miles  and  also  has  the  standard  7.7  kg  warhead 
found  in  the  Hydra  70  rocket.  The  big  difference  between  APKWS  and  LOGIR  is  that 
APKWS  uses  semi-active  laser  guidance.  There  is  a  big  difference  between  APKWS  and 
other  laser  guided  rockets  in  that  it  uses  four  distributed  aperture  semi-active  laser 
seekers  (DASALs),  similar  to  those  found  in  the  Army’s  precision  guided  mortar 
munitions  program.  With  these  four  seekers,  the  APKWS  has  demonstrated  an  accuracy 
of  0.5  meters  in  tests  (Jane’s  2010).  Eight  operational  assessment  test  firings  were 
conducted  by  the  U.S.  Marines  in  January  2010  and  in  April  2010,  low  rate  initial 
production  was  approved.  As  with  the  LOGIR,  there  is  a  good  chance  that  ONR  will 
attempt  to  utilize  the  existing  Hydra  70  rocket  launchers,  which  come  in  seven  packs  and 
19  packs  of  rockets.  The  APKWS  is  very  similar  to  the  LOGIR  in  capability  but  appears 
to  be  slightly  more  accurate  but  with  a  slower  rate  of  fire  due  to  its  required  lasing  of  the 
target. 
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6. 


Directional  Attack  Guided  Rocket  (DAGR) 


The  DAGR  is  another  attempt  to  convert  the  conventional  Hydra  70  and  the 
standard  CRV7  rockets  into  precision-guidance  weapons.  Like  the  APKWS,  the  DAGR 
uses  a  semi-active  laser  (SAL)  seeker.  The  DAGR  is  so  similar  to  the  Hellfire  that  it  is 
often  called  the  Hellfire  II.  A  feature  about  the  DAGR  that  stands  out  is  that  Lockheed 
Martin  has  designed  a  smart  launcher  for  the  DAGR  that  can  be  clipped  into  place  on  a 
M299  or  M3 10  Hellfire  launcher.  This  launcher  allows  for  a  four-pack  of  DAGRs  to  be 
placed  in  each  Hellfire  slot.  Lockheed  Martin  is  also  working  on  a  six-pack  DAGR 
launcher  design.  The  DAGR  is  highly  capable  precision  rocket,  having  both  Lock  On 
Before  Launch  (LOBL)  or  Lock  On  After  Launch  (LOAL)  modes.  If  launched  from  an 
altitude  of  20,000  feet,  it  is  designed  to  be  accurate  to  within  one  meter  at  a  12,000  meter 
range.  If  launched  from  near-ground  level  it  is  accurate  to  within  one  meter  at  7,000 
meters.  The  DAGR  is  currently  being  looked  at  by  all  branches  of  the  U.S.  armed  forces. 


Figure  6.  A  four-round  DAGR  pod  plus  a  Hellfire  missile  on  a  M3 10  launcher 

(From  Jane’s  2010) 


G.  RELATED  STUDIES 

Several  theses  have  been  conducted  at  the  Naval  Postgraduate  School  concerning 

LCS  and/or  allied  ships  conducting  operations  against  large  numbers  of  small  boats. 

Andre  Tiwari’s  thesis  (2008)  showed  that  a  gap  in  capability  exists  in  the  surface  force  to 

defend  itself  against  small  threat  craft.  Benjamin  Abbot’s  thesis  (2008)  explored  the  best 

mix  of  LCS  mission  packages  and  determined  that  LCS  should  operate  in  squadrons  of 

between  six  to  ten  ships  for  the  best  results,  with  five  LCS  equipped  for  the  primary 

threat  and  two  LCS  for  the  secondary  threat.  Another  thesis  dealing  with  LCS  was  done 

by  Michael  Milliken  in  September  2008  in  which  he  conducted  an  impact  analysis  of  a 

mixed  squadron,  containing  LCS  and  multi-mission  surface  platforms,  on  blue  force 
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casualties  and  mission  effectiveness.  Milliken’s  conclusion  was  that  a  squadron  with  five 
to  eleven  LCSs  with  one  to  two  DDGs  is  the  most  effective  in  an  SUW  scenario.  Omur 
Ozdemir’s  thesis  (2009)  did  a  comparison  of  the  Freedom  class  LCS  and  other 
frigates/corvettes  against  small  boat,  fast  patrol  boat,  and  submarine  threats  in  confined 
waters.  His  conclusion  was  that  the  LCS  was  the  most  combat  effective,  but  that  its  high 
cost  meant  that  it  was  not  the  best  candidate. 

H.  SUMMARY 

The  LCS  is  the  Navy’s  answer  to  the  challenge  of  projecting  power  “from  the 
sea”  in  the  littoral  environment  and  in  supporting  ground  forces  ashore.  The  concept  of 
operations  calls  for  the  LCS  to  operate  distributed  sensor  platfonns,  both  manned  and 
unmanned  to  find  and  engage  threats,  keeping  the  LCS  out  of  direct  combat  and  utilizing 
missiles  from  the  LCS  matched  with  the  sensor  platforms  to  provide  over-the-horizon 
capability. 


18 


III.  MODEL  DEVELOPMENT 


A.  INTRODUCTION 

In  order  to  accurately  capture  how  these  different  missiles  will  work  in  a  wartime 
environment,  a  robust  scenario  that  contains  a  realistic  and  capable  threat  is  required.  In 
this  chapter,  a  brief  description  of  the  MANA  simulation  tool  is  provided  as  is  a 
description  of  the  behavior  of  the  simulation  model. 

B.  THE  MANA  COMBAT  SIMULATION  TOOL 

1.  Choosing  MANA 

MANA  is  combat  model  developed  and  provided  to  NPS  by  New  Zealand’s 
Defense  Technology  Agency  (DTA).  “MANA  is  an  agent-based  distillation  model 
developed  by  DTA  for  use  in  military  operations  analysis  studies”  (McIntosh  2009).  One 
of  the  best  qualities  of  MANA  and  one  of  the  main  reasons  it  was  used  in  this  thesis  is 
that  it  is  event  driven,  and  as  a  result,  it  gives  a  remarkable  depiction  of  simulated 
combat.  In  using  MANA,  one  can  more  accurately  depict  the  attributes  of  the  individual 
agents  and  MANA  gives  one  the  ability  to  vary  these  attributes  which  allows  the 
simulator  to  have  the  ability  to  observe  and  quantify  the  effects  of  these  varying  attributes 
on  the  battlefield  outcomes.  Another  point  in  MANA’s  favor  is  how  easy  it  is  to  use.  It 
has  a  very  simple  interface  that  allows  the  simulator  to  vary  all  of  the  attributes  of  the 
agents  involved.  “The  simple  nature  of  the  model  allows  both  rapid  parameter  space 
exploration  and  experimentation  with  co-evolving  tactics,  yet  it  has  enough  sophistication 
to  produce  realistic  looking  behaviors  and  tactics  (Lauren  2002).” 

2.  MANA  Characteristics 

In  this  thesis,  version  5.00.89  of  MANA  is  being  used.  Released  in  June  2010, 
this  is  the  latest  version  of  MANA.  One  of  the  biggest  advantages  of  MANA  version  5 
over  older  versions  is  that  battlefield  distances,  agent  speeds,  and  weapons  characteristics 
can  be  defined  in  real  world  units  (for  example,  meters,  km/hr,  nautical  miles).  This  is 
possible  because  the  cell-based  movement  scheme  of  previous  MANA  versions  has  been 
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replaced  by  a  vector-based  scheme.  Not  only  does  this  allow  for  real  world  units  to  be 
used,  but  it  also  allows  for  much  larger  battlefields  to  be  utilized  as  well.  One  of  the 
other  advantages  of  MANA  in  general,  is  that  it  leaves  out  detailed  physical  attributes  of 
the  entities  being  analyzed,  such  as  the  effects  of  the  sea  state  upon  the  ships  involved, 
allowing  for  the  model  to  run  relatively  fast  on  a  PC  or  laptop.  This  also  means  that  a 
large  number  of  runs  of  the  scenario  with  varying  attributes  can  be  explored  in  a 
reasonable  amount  of  time.  In  MANA  the  user  develops  squads,  one  for  each  type  of 
platform  being  used  in  the  simulation.  Each  squad  is  assigned  weapons  and  sensors  and 
personalities  based  on  the  user’s  needs.  These  personalities  give  the  squads  simple  rules 
about  how  they  are  to  move  based  upon  the  location  of  other  squads  and  conditions  on 
the  battlefield.  The  user  sets  this  up  by  weighing  different  aspects  of  the  squad  in  the 
Personalities  tab  of  the  squad  properties,  shown  below  in  Figure  7.  In  this  example  the 
squad  would  be  focused  only  on  going  to  the  next  waypoint,  but  would  fire  upon  enemies 
if  they  are  within  range  and  its  weapons  are  activated,  but  would  not  pursue  any  action 
other  than  following  its  pre-designed  path. 


Figure  7.  The  personality  screen  used  in  MANA  version  5. 
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MAN  A  V 


DEFENCE  TECHNOLOGY  AGENCY 


Vector-Based  Movement 

Map  Aware  Non-uniform  Automata 


This  software  has  been  developed  by  Gregory  McIntosh.  David  Galliqan. 

Mark  Anderson  and  Michael  Lauren  for  the  New  Zealand  Army  and  Defence  Force. 

Terms  of  use: 

This  software  belongs  to  the  Defence  Technology  Agency  (DTA),  New  Zealand 
Defence  Force.  It  is  not  to  be  distributed  without  the  permission  of  DTA.  This 
product  is  not  to  be  offered  for  resale. 


See  http:Wzdf.mil.nzy  mana  for  latest  updates  and  support. 

I  understand  I  can  only  use  this  software  subject  to  the  conditions  stated  above. 


Version  5.00.89 


Figure  8.  Screen  shot  of  MANA  start  up  screen. 


C.  CHARACTERISTICS  OF  THE  SIMULATION  MODEL 

The  focus  of  this  section  is  to  provide  the  characteristics  of  the  MANA  model  that 
was  created  for  this  research.  The  goal  of  the  simulation  is  discussed  as  is  the  scale,  the 
friendly  forces,  the  enemy  forces,  and  issues  regarding  sources  of  data,  abstractions,  and 
assumptions.  A  more  detailed  breakdown  of  the  personalities  and  capabilities  of  the 
friendly  and  enemy  forces  can  be  found  in  Appendix  A. 

1.  Goal  of  the  Simulation 

The  scenario  used  in  this  thesis  is  designed  to  stress  the  capabilities  and  potential 
capabilities  of  each  missile  type  in  order  to  gain  insight  into  which  missiles  would  be  best 
for  use  with  the  SUW  module  equipped  LCS.  The  factors  that  play  an  important  role  in 
this  simulation  are  all  concerned  with  the  missile  capabilities.  These  factors  are  the 
maximum  range,  the  rate  of  fire,  the  probability  of  hit,  and  the  number  of  missiles  being 
carried.  Using  design  of  experiment  techniques,  these  capabilities  are  explored  over  large 
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ranges  to  determine  which  were  the  most  important  and  to  what  extent  and  which  missile, 
based  on  its  potential  capabilities,  is  the  best  for  use.  Chapter  IV  describes  the 
experimental  design  in  detail. 

2.  Scale  and  Terrain 

MANA  is  a  time  step  model.  For  this  study,  the  model  was  set  so  that  each  time 
step  was  equal  to  10  seconds  of  real  world  time.  Each  run  of  the  simulation  lasts  no 
longer  than  500  time  steps;  which  is  equivalent  to  about  an  hour  and  23  minutes  in  real 
world  time.  The  simulation  map  is  40  kilometers  by  40  kilometers.  MANA  does  provide 
for  one  to  be  able  to  model  different  types  of  terrain.  Since  the  effect  of  the  sea  state  on 
the  ships  was  not  one  of  the  aspects  being  measured,  no  terrain  was  used  in  this 
simulation  as  it  would  be  introduce  unnecessary  detail. 

3.  Friendly  Forces 

The  friendly  forces  are  assigned  a  home  position  as  well  as  waypoints.  The  LCS 
transits  from  its  home  position  due  north  and  engages  enemies  when  it  is  capable.  The 
helicopter/UAV  is  25  kilometers  northeast  by  east  of  the  LCS  and  is  transiting  towards 
the  suspected  location  of  enemy  guided  missile  patrol  boats.  The  helicopter/UAV  transits 
according  to  its  speed  and  will  pursue  and  engage  enemies  detected. 

4.  Enemy  Forces 

Just  like  the  friendly  forces,  the  enemy  forces  are  also  assigned  a  home  position. 
Their  home  position  is  a  box  in  which  they  can  start  at  any  point  in  that  box,  which 
MANA  decides  randomly.  They  know  there  is  an  American  LCS  to  the  west  and  are 
headed  to  get  good  contact  on  it  and  attack.  The  enemy  threat  consists  of  20  missile 
boats.  They  transit  and  attack  as  a  group  for  safety  and  cumulative  strength.  Once  the 
LCS  is  detected  the  missile  boats  will  pursue. 
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5.  Sources,  Abstractions,  and  Assumptions 

As  with  every  simulation,  the  source  of  input  data  and  assumptions  are  important. 
In  this  scenario,  communications  and  logistics  are  assumed  to  work  perfectly,  and  fuel  is 
unlimited.  Maintenance  and  equipment  failure  are  not  considered. 

Enemy  force  sensor  and  weapon  information,  number  of  weapons  per  enemy 
agent,  and  the  capabilities  of  some  of  the  friendly  sensors  and  weapons  were  taken  from 
Jane’s  Fighting  Ships  2010.  The  probabilities  associated  with  the  sensors  and  weapons 
were  generalized  and  reviewed  by  Jeff  Kline,  retired  Navy  Captain  and  Chair  of  Warfare 
Innovation  at  NPS,  and  LCDR  Stacey  Prescott,  a  SH-60B  pilot  with  Hellfire  experience. 
These  probabilities  are  explored  through  design  of  experiment  techniques  that  are 
discussed  in  the  next  chapter. 

It  is  a  rare  occurrence  in  which  a  simulation  tool  perfectly  fits  the  problem  being 
modeled.  Oftentimes,  issues  are  discovered  during  the  model  development  process  that 
are  either  fixed  by  the  developers  or  addressed  via  other  modeling  work-arounds.  In  this 
thesis,  one  such  modeling  issue  was  discovered.  The  issue  had  to  do  with  MANA’s 
ability  to  simulate  one  squad’s  ability  to  spoof  a  missile  using  electronic  warfare  or  chaff, 
or  to  be  able  to  shoot  it  down.  Since  the  enemy  missile  boats  that  were  simulated  are  not 
equipped  with  an  anti-missile  defense  system,  it  was  not  necessary  to  worry  about  this 
issue  with  them.  With  the  focus  solely  on  the  LCS  it  was  easy  to  ignore  the  problem  by 
making  the  goal  of  the  friendly  forces  to  be  to  ensure  that  none  of  the  missile  boats  even 
got  to  where  they  could  fire  their  missiles  at  the  LCS. 

D.  SUMMARY 

MANA  was  used  for  this  thesis  to  simulate  a  scenario  in  which  a  SUW  module 
equipped  LCS  and  its  accompanying  aircraft  are  faced  with  20  inbound  missile  boats. 
The  scenario  is  designed  to  test  the  potential  capabilities  of  different  surface-to-surface 
and  air-to-surface  missiles  that  the  SUW  module  may  be  equipped  with  and  to  determine 
which  capabilities  are  the  most  important. 
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IV.  EXPERIMENT  DESIGN 


A.  INTRODUCTION 

This  chapter  covers  the  assumptions  used  for  the  setup  of  the  simulation  as  well  as 
a  description  of  how  the  simulation  was  designed.  Special  attention  is  paid  to  the 
parameters  used  to  direct  the  abilities  and  actions  of  the  squads. 

B.  VARIABLES  OF  INTEREST 

There  are  two  types  of  variables  that  are  used  in  simulation:  controllable  and 
uncontrollable.  Controllable  variables  are  those  that  can  be  altered  by  the  decision  maker 
and  uncontrollable  variables  are  the  ones  that  a  decision  maker  cannot  control. 
Controllable  variables  are  often  known  as  decision  factors,  whereas  uncontrollable 
variables  are  often  referred  to  as  noise  factors.  This  thesis  focuses  on  the  decision  factors 
in  order  to  provide  greater  insight  into  which  weapons  systems  provide  the  best  option  in 
Surface  Warfare.  Since  the  enemy  sensor  and  weapon  ranges,  their  probabilities  of 
detection  and  kill,  and  the  number  of  enemy  patrol  craft  are  fixed,  there  are  no  noise 
factors  in  this  thesis.  Table  2  defines  the  variables  used  in  this  study. 


Table  2.  Variable  factors  used  in  the  experiment  design. 


Factor 

Value  Range 

Explanation 

LCS  Probability  of 

Detecti 
on  (PD) 

0.5... 1 

Probability  of  Detection  associated  with 
the  LCS  seaframe 

MH-60R  PD 

0.5.. .1 

Probability  of  Detection  associated  with 
the  MH-60  sensor 

UAV  PD 

0.4... 1 

Probability  of  Detection  associated  with 
the  UAV  sensor 

NLOS  Probability  of 
Kill 
(PK) 

0.13. ..0.585 

Probability  of  kill  associated  with  the 
NLOS  Missile  System 

NLOS  Inter-firing  rate 
(Ifr) 

5. ..25 

Time  between  subsequent  firings  of  the 
NLOS  Missile  System 

Harpoon  PK 

0.8075. ..0.9405 

Probability  of  kill  associated  with  the 
Harpoon  missile  system 
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Factor 

Value  Range 

Explanation 

Harpoon  Ifr 

10. ..25 

Time  between  subsequent  firings  of  the 
Harpoon  Missile  System 

Hellfire  Pk 

0.6375. ..0.7125 

Probability  of  kill  associated  with  the 
Hellfire  missile 

Hellfire  Ifr 

8... 18 

Time  between  subsequent  firings  of  the 
Hellfire  missile 

Hellfire  Quantity 

Carried 

(QC) 

8  MH60 

2... 6  UAV 

Number  of  Hellfires  carried  in  a  given 
run 

Hellfire  Maximum 

Effectiv 
e  Range 
(MER) 

5000... 8000 

Maximum  effective  range  of  the  Hellfire 
missle  in  a  given  run 

LOGIR  Pk 

0.4225. ..0.585 

Probability  of  kill  associated  with  the 
LOGIR 

LOGIR  Ifr 

6. ..16 

Time  between  subsequent  firings  of  the 
LOGIR 

LOGIR  Qc 

14.. . 38  MH-60 

7.. .  19  UAV 

Number  of  LOGIR  carried  in  a  given  run 

LOGIR  MER 

4000... 5800 

Maximum  effective  range  of  LOGIR  in  a 
given  run 

APKWS  Pk 

0.4875. ..0.6175 

Probability  of  kill  associated  with  the 
APKWS 

APKWS  Ifr 

8. ..20 

Time  between  subsequent  firings  of  the 
APKWS 

APKWS  Qc 

14.. . 38  MH-60 

7.. .  19  UAV 

Number  of  APKWS  carried  in  a  given 
run 

APKWS  MER 

4000... 5000 

Maximum  effective  range  of  APKWS  in 
a  given  run 

DAGR  Pk 

0.4875. ..0.6175 

Probability  of  kill  associated  with  the 
DAGR 

DAGR  Ifr 

8. ..20 

Time  between  subsequent  firings  of  the 
DAGR 

DAGR  Qc 

8.. .  12  MH60 

2.. . 8  UAV 

Number  of  DAGR  carried  in  a  given  run 

DAGR  MER 

4500... 7000 

Maximum  effective  range  of  DAGR  in  a 
given  run 

1.  Controllable  Factors 

The  following  variables  are  chosen  in  order  to  explore  the  effectiveness  of 

different  surface-to-surface  and  air-to-surface  weapon  systems. 
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a.  LCS  Probability  of  Detection  (Pd) 

The  probability  of  detection  associated  with  the  sensor  used  by  the  LCS. 
The  sensor  being  modeled  is  the  3D  surface  search  radar  that  will  be  used  by  LCS. 

b.  MH-60R  Pd 

The  probability  of  detection  associated  with  the  sensor  used  by  the  MH- 
60R.  The  sensor  being  modeled  is  the  AN/APS-147  surface  search  radar. 

c.  UAVPd 

The  probability  of  detection  associated  with  the  sensor  used  by  the  MQ-8 
Fire  Scout.  The  sensor  being  modeled  is  the  Tactical  Synthetic  Aperture  Radar  (TSAR). 

d.  NLOS  Probability  of  Kill  (Pk) 

The  probability  of  kill  associated  with  the  NLOS  missile  system  when 
employed  in  the  SUW  mission  package.  The  Pk  is  the  product  of  the  probability  of  hit 
and  the  probability  of  kill  given  a  hit. 

e.  NLOS  Inter  Firing  Rate  (Ifr) 

The  inter-firing  rate  associated  with  the  NLOS  system.  This  is  the  amount 
of  time  between  subsequent  firings  of  the  NLOS. 

f  Harpoon  Pk 

The  probability  of  kill  associated  with  the  Harpoon  missile  system  when 
employed  in  the  SUW  mission  package.  The  Pk  is  the  product  of  the  probability  of  hit 
and  the  probability  of  kill  given  a  hit. 

g.  Harpoon  Ifr 

The  inter-firing  rate  associated  with  the  Harpoon  missile.  This  is  the 
amount  of  time  between  subsequent  firings  of  the  Harpoon  missile. 
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h.  Hellfire  Pk 

The  probability  of  kill  associated  with  the  Hellfire  missile  when  employed 

by  the  MH-60R  or  the  Fire  Scout  UAV.  The  Pk  is  the  product  of  the  probability  of  hit 
and  the  probability  of  kill  given  a  hit. 

i.  Hellfire  Ifr 

The  inter-firing  rate  associated  with  the  Hellfire  missile.  This  is  the 
amount  of  time  between  subsequent  firings  of  the  Hellfire  missile. 

j.  Hellfire  Quantity  Carried  (Qc) 

The  number  of  Hellfire  missiles  being  carried  by  the  MH-60R  or  the  Fire 
Scout  UAV.  This  is  based  off  of  it  being  known  that  the  MH-60R  is  capable  of  carrying 
eight  Hellfire  Missiles  (Jane’s)  and  that  the  Fire  Scout  is  capable  of  carrying  two  based 
on  its  weight  limitations. 

k.  Hellfire  Maximum  Effective  Range  (MER) 

The  maximum  effective  range  of  the  Hellfire  missile  when  employed  by 
the  MH-60R  or  the  Fire  Scout  UAV. 

l.  LOGIR  Pk 

The  probability  of  kill  associated  with  the  LOGIR  when  employed  by  the 
MH-60R  or  the  Fire  Scout  UAV.  The  Pk  is  the  product  of  the  probability  of  hit  and  the 
probability  of  kill  given  a  hit. 

m.  LOGIR  Ifr 

The  inter-firing  rate  associated  with  the  LOGIR.  This  is  the  amount  of 
time  between  subsequent  firings  of  the  LOGIR. 

n.  LOGIR  Qc 

The  number  of  LOGIR  being  carried  by  the  MH-60R  or  Fire  Scout  UAV. 


28 


o.  LOGIR  MER 

The  maximum  effective  range  of  the  LOGIR  when  employed  by  the  MH- 
60R  or  the  Fire  Scout  UAV. 

p.  APKWSPk 

The  probability  of  kill  associated  with  the  APKWS  when  equipped  by  the 
MH-60R  or  the  Fire  Scout  UAV.  The  Pk  is  the  product  of  the  probability  of  hit  and  the 
probability  of  kill  given  a  hit. 

q.  APKWS  Ifr 

The-inter  firing  rate  associated  with  the  APKWS.  This  is  the  amount  of 
time  between  subsequent  firings  of  the  APKWS. 

r.  APKWS  Qc 

The  number  of  APKWS  being  carried  by  the  MH-60R  or  Fire  Scout  UAV. 

.s.  APKWS  MER 

The  maximum  effective  range  of  the  APKWS  when  employed  by  the  MH- 
60R  or  the  Fire  Scout  UAV. 

t.  DAGRPk 

The  probability  of  kill  associated  with  the  DAGR  when  employed  by  the 
MH-60R  or  the  Fire  Scout  UAV.  The  Pk  is  the  product  of  the  probability  of  hit  and  the 
probability  of  kill  given  a  hit. 

u.  DAGR  Ifr 

The  inter-firing  rate  associated  with  the  DAGR.  This  is  the  amount  of 
time  between  subsequent  firings  of  the  DAGR. 

v.  DAGR  Qc 

The  number  of  DAGR  being  carried  by  the  MH-60R  or  Fire  Scout  UAV. 
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DAGRMER 


w. 

The  maximum  effective  range  of  the  DAGR  when  employed  by  the  MH- 
60R  or  the  Fire  Scout  UAV. 

2.  Assumptions 

•  The  DAGR,  APKWS,  and  LOGIR  will  be  based  on  the  Hydra  70  rocket 
with  its  7.7  kg  warhead  and  not  the  CRV7  rocket  with  4.5  kg  warhead. 

•  The  Fire  Scout  UAV  can  carry  two  Hellfire  Missiles 

•  The  Fire  Scout  UAV  can  laser  designate  targets  for  the  LCS 

•  LOGIR  and  DAGR  have  the  same  minimum  rage  as  APKWS,  which  is 
reported  as  0.93  miles  in  Jane’s 

•  The  LCS  can  fire  its  missiles  off  of  MH-60R  or  Fire  Scout  sensor  data 

•  The  MH-60R  detect  range  of  a  50  foot  missile  patrol  boat  is  30,000  meters 
and  the  classify  range  is  8,500  meters 

•  The  Fire  Scout  detect  range  of  a  50  foot  missile  patrol  boat  is  28,000 
meters  and  the  classify  range  is  8,000  meters 

C.  THE  EXPERIMENT 

1.  The  Nearly  Orthogonal  Latin  Hypercube  (NOLH) 

The  NOLH  experimental  design  technique  was  developed  at  NPS  by  Lt.  Col. 
Thomas  Cioppa,  United  States  Army,  in  2002.  This  technique  was  designed  to  efficiently 
explore  simulations  that  have  a  large  input  space,  requiring  minimum  a  priori 
assumptions  (Cioppa,  2002).  The  space  filling  property  of  the  NOLH  allows  the  analyst 
to  explore  more  of  the  input  space  than  the  traditional  factorial  design,  in  which  only  high 
and  low  values  are  considered.  The  NOLH  does  not  allow  the  analyst  to  see  all  of  the 
response  surface,  but  does  enable  the  analyst  to  see  a  broader  section  of  the  response 


30 


surface.  A  NOLH  generation  tool  created  by  Professor  Susan  Sanchez  at  NPS  was  used 
to  generate  the  designs  for  this  thesis.  Detailed  tables  of  the  experimental  designs  used 
are  provided  in  Appendix  B. 

2.  Exploratory  Design 

To  explore  MANA’s  suitability  to  address  the  question  posed  by  this  thesis,  an 
exploratory  design  of  the  scenario  was  created.  This  scenario  is  very  abstract,  includes  a 
smaller  number  of  threat  vessels,  and  is  intended  to  provide  insight  into  the  modeling  of 
the  different  personalities,  sensor  capabilities,  and  communications  capabilities  for  each 
squad  in  the  scenario.  This  scenario  verifies  that  the  aircraft  are  not  able  to  be  shot  down 
by  the  surface-to-surface  missiles  on  the  red  forces,  that  the  LCS  can  use  the  aircraft’s 
sensor  data,  and  that  the  sensors  and  weapons  are  working  properly.  Sensor  verification 
was,  in  part,  accomplished  by  having  MANA  track  the  detection  of  each  squad  by  the 
other  squads  and  using  this  data  to  tweak  the  personalities  of  the  squads. 

3.  Preliminary  Design 

With  the  exploratory  design  working  bug  free,  it  was  time  to  expand  upon  it.  The 
aircraft  was  copied  and  made  into  two  squads  with  the  adjustments  made  to  differentiate 
between  the  Fire  Scout  UAV  and  the  MH-60R.  The  simulation  was  then  run  several 
times  with  the  aircraft  variously  turned  on  or  off  to  ensure  that  the  switch  was  working 
smoothly.  Turning  the  aircraft  on  and  off  was  done  by  marking  the  corresponding 
aircraft  squad  as  either  active  or  inactive  in  that  particular  scenario.  The  same  procedure 
was  done  with  the  aircrafts’  weapons.  Then,  the  number  of  enemy  missile  patrol  boats 
was  slowly  worked  up  in  increments  of  two  until  reaching  the  maximum  of  20. 

4.  Final  Design 

After  several  dozen  runs  of  the  preliminary  design,  during  which  the  design  was 
validated  with  the  assistance  of  Mary  McDonald  of  the  SEED  center,  the  final  design  was 
implemented.  The  512  runs  created  by  the  NOLH  were  used  with  each  run  being 
replicated  40  times  for  both  versions  of  the  scenario,  one  with  the  Fire  Scout  UAV  active 
and  one  with  the  MH-60R  active.  These  40  replications  of  each  run  resulted  in  there 
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being  20,480  runs  for  each  scenario,  which  resulted  in  532,480  data  points  for  each 
scenario  and  1,064,960  total  data  points.  The  analysis  of  these  data  points  is  the  basis  for 
this  thesis  and  is  covered  in  the  next  chapter. 

D.  RUNNING  THE  EXPERIMENT 

The  base  case  MANA  scenario,  in  Extensible  Markup  Language  (XML)  fonnat, 
and  the  DOE  file,  in  comma-separated  value  (CSV)  format,  were  entered  into  a  software 
program  called  XStudy,  written  by  SEED  Center  Research  Associate  Steve  Upton.  The 
XStudy  program  enables  the  user  to  map  each  column  in  the  design  file  to  a  specific 
parameter  in  MANA  using  XPath  expressions.  Other  details  about  the  study  design,  such 
as  the  version  of  MANA  and  number  of  replications  per  design  point,  are  also  entered 
into  this  tool,  yielding  a  single  Study.xml  file.  This  file  is  used  by  another  program  called 
oldmcdata,  also  written  by  Steve  Upton,  which  automatically  updates  the  MANA  XML 
file,  producing  a  separate  XML  scenario  file  for  each  of  the  different  factor  combinations. 
This  program  then  launches  MANA  runs  on  the  SEED  Center's  high-perfonnance 
computing  cluster  for  each  of  the  separate  files.  This  is  done  to  automate  the  parallel 
implementation  of  the  MANA  simulated  runs  and  subsequently  collect  the  output  data 
into  a  single  CSV  file. 

E.  SUMMARY 

The  NOLH  design  provided  by  Professor  Susan  Sanchez  of  NPS  was  used  to  vary 
the  23  variables  across  the  full  range  of  values  into  a  total  of  1,028  rows  of  data.  These 
rows  of  data,  in  two  sets  of  512  each,  were  each  executed  40  times  to  provide  1,064,960 
total  data  points. 
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V.  DATA  ANALYSIS 


The  experiment  described  in  the  Chapter  IV  generated  a  large  amount  of  data. 
This  chapter  begins  by  discussing  how  the  data  was  collected  and  processed  for  analysis. 
The  purpose  of  this  analysis  is  to  provide  insight  into  the  research  questions,  which  are 
restated  in  this  chapter. 

A.  DATA  COLLECTION  AND  PROCESSING 

The  output  provided  by  MANA  is  in  the  form  of  a  CSV  file  that  allows  for  simple 
processing,  as  it  is  a  file  that  can  be  read  by  a  multitude  of  statistical  software  programs 
without  the  need  to  adjust  the  data.  These  output  files  provide  the  number  of  casualties  to 
each  squad,  as  well  as  the  input  variables  that  are  used  with  each  run.  For  each  scenario 
there  were  512  different  sets  of  input  data  that  were  run  40  times  each,  resulting  in  20,480 
rows  of  data  for  each  scenario,  or  40,960  rows  of  data  total.  In  order  to  compile  the 
output  data  into  a  more  manageable  number,  summaries  of  the  output  files  were  needed. 
Each  of  the  scenario  output  files  was  imported  into  a  statistical  software  packaged  called 
JMP  version  8.0,  a  program  created  by  SAS  Institute  Incorporated.  The  means  and 
standard  deviations  were  generated  for  each  input  combination,  bringing  the  rows  of  data 
from  20,480  down  to  512  for  the  summaries  of  the  two  scenarios.  The  measure  of 
effectiveness  (MOE)  used  in  this  research  is  the  mean  total  Red  casualties. 

B.  INSIGHTS  INTO  RESEARCH  QUESTIONS 

In  Chapter  I,  three  questions  were  offered  as  the  basis  of  this  research.  Each  of 
these  questions  has  been  addressed  through  data  analysis.  The  research  questions  for  this 
thesis  are: 

•  What  surface-to-surface  missile  should  LCS  deploy  with? 

•  What  type  of  air-to-surface  missiles  should  the  aircraft  be  equipped  with 
to  be  the  most  effective? 

•  What  factors  are  more  important  in  determining  the  type  of  missile  to 
select? 
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This  analysis  includes  the  use  of  several  analytical  tools,  including  partition  trees 
and  bivariate  analysis. 

1.  Surface-to-Surface  Missile  for  the  SUW  Module 

The  question  regarding  the  LCS  surface-to-surface  missile  is  the  same  in  both 
versions  of  the  scenario  and  so  data  from  both  output  summaries  will  be  viewed. 

a.  MH-60R  Scenario 

As  shown  in  Table  3,  the  first  split  is  determined  by  whether  or  not  the 
ship  is  equipped  with  NLOS  or  Harpoon.  With  NLOS  the  mean  number  of  Red 
casualties  is  10.70  while  the  mean  with  Harpoon  is  only  4.98.  This  result  can  be 
interpreted  to  mean  that  the  NLOS  missile  system  is  superior,  but  if  one  splits  the  data 
further,  one  can  see  that,  when  the  rate  of  fire  of  the  NLOS  is  less  than  1 1  (or  greater  than 
9.1  seconds  between  shots),  that  the  mean  number  of  Red  casualties  drops  to  4.12  making 
it  less  capable  than  Harpoon.  As  one  can  see  from  Table  3,  the  Pk  of  the  NLOS  is  less 
important  as  long  as  the  rate  of  fire  is  1 1  or  better  (9.1  seconds  or  less  between  shots).  If 
the  NLOS  rate  is  below  11,  then  the  Harpoon  appears  to  be  a  better  choice  based  on  the 
split  as  long  as  the  Harpoon  rate  of  fire  is  greater  or  equal  to  6  (16.7  seconds  between 
shots  of  less). 
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Table  3. 


The  partition  tree  comparing  the  Harpoon  and  NLOS  data  from  the  MH- 
60R  summary  data,  based  upon  the  mean  number  of  Red  casualties. 


b.  UA  V  Scenario 

As  shown  in  Table  4,  the  data  from  the  UAV  runs  shows  nearly  identical 
results  as  the  MH-60R  data.  The  NLOS  is  superior  to  the  Harpoon  as  long  as  the  rate  is 
greater  than  or  equal  to  11  (9.1  seconds  or  less  between  shots),  with  there  being  a  vast 
difference  between  the  means  (10.21  vs  4.02).  If  the  NLOS  rate  is  greater  than  11,  then  it 
does  not  appear  to  matter  much  what  the  Pk  is.  As  before,  the  Harpoon  at  a  rate  greater 
than  or  equal  to  6  (16.7  seconds  between  shots  of  less)  is  superior  to  the  NLOS  with  a 
rate  less  than  11  (greater  than  9.1  seconds  between  shots),  but  only  if  the  NLOS  Pk  is 
below  0.533. 
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Table  4.  The  partition  tree  comparing  the  Harpoon  and  NLOS  data  from  the  UAV 

summary  data,  based  upon  the  mean  number  of  Red  casualties. 


c.  Overall 

The  partition  trees  for  both  summaries  look  very  similar  and  both  agree 
that  the  NLOS  is  superior  to  the  Harpoon  as  long  as  the  rate  of  fire  is  9.1  seconds 
between  shots  or  less.  If  the  NLOS  rate  of  fire  is  less  than  9.1  seconds,  then  a  Harpoon 
with  a  rate  of  16.7  seconds  or  less  between  shots  is  slightly  superior. 

2.  What  Type  of  Air-to-Surface  Missile  Is  Most  Effective 

The  question  regarding  which  air-to-surface  missile  would  be  best  to  equip  the 
aircraft  with  is  the  same  in  both  versions  of  the  scenario  and  so  data  from  both  output 
summaries  will  be  viewed. 
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a.  MH-60R  Scenario 

Table  5  shows  that  the  LOGIR  is  the  most  effective  missile  system.  The 
LOGIR  had  a  mean  of  9.30,  DAGR  had  a  mean  of  7.56,  APKWS  had  a  mean  of  6.92,  and 
Hclllire  had  a  mean  of  6.45.  Unfortunately,  this  interpretation  does  not  provide  a 
complete  picture  since  the  Standard  Deviations  are  approximately  six  in  each  category. 


Table  5.  The  partition  tree  of  the  MH-60R  summary  data  in  which  the  number  of 

mean  Red  casualties  based  on  the  missiles  equipped  are  the  only  categories  of 

data  studied. 
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As  is  evidenced  by  Table  6,  the  first  split  still  takes  place  with  the  aircraft 
being  equipped  with  LOGIR  having  a  mean  of  9.30  compared  to  a  mean  of  6.95  for  all 
other  missile  types.  But,  if  the  rate  of  fire  of  the  LOGIR  is  less  than  1 1  (or  greater  than 
9.1  seconds  between  shots),  then  it  is  no  longer  the  best  option.  The  next  missile  that 
meets  the  criterion  for  a  split  is  the  DAGR.  If  the  DAGR  is  equipped  and  its  rate  is 
greater  than  11  (9.1  seconds  or  less  between  shots)  then  it  is  the  third  best  option,  with  a 
mean  of  12.418,  compared  to  the  LOGIR  with  a  rate  greater  than  11  having  a  mean  of 
13.366.  The  APKWS  comes  out  ahead  after  the  breakdown  when  it  has  a  rate  greater 
than  11,  beating  out  LOGIR  with  a  mean  of  14.043.  Of  the  two,  however,  LOGIR  is 
more  likely  to  have  a  higher  rate  of  fire,  being  a  fire-and-forget  type  of  missile.  The  next 
section  will  feature  a  more  in-depth  look  at  what  features  affect  the  missiles  the  most. 


Table  6.  From  the  MH-60R  Summary  data.  A  partition  tree  of  the  four  air-to- 

surface  missile  options  and  all  of  their  input  options. 


T  All  Rows 

Count 

512 

LogWorth 

Difference 

Mean 

7.5908203 

3.994659 

2.34873 

Std  Dev 

6.109146 

”  LOGIR(No) 

Count 

372 

LogWorth 

Difference 

Mean 

6.9485887 

0.7005972 

0.89115 

Std  Dev 

5.9636994 

-LOGIRCres) 

Count 

140 

LogWorth 

Difference 

Mean 

9.2973214 

60.81009 

9.81986 

Std  Dev 

6.1838475 

-DAGRINo) 

Count 

256 

LogWorth 

Difference 

Mean 

6.6707031 

0.2592269 

0.47212 

Std  Dev 

5.8403638 

*  DAGR(Yes) 

Count  116 

LogWorth 

Difference 

Mean 

7.5618534 

9.7020595 

6.86917 

Std  Dev 

6.20902 

w  LOGIRRt<1 1 

Count  58 

Mean  3.5456897 

Std  Dev  3.647906 

”LOGIRRt>=11 

Count  82 

Mean  13.365549 

Std  Dev  3.9767721 

►  Candidates 

►  Candidates 

*  HellfirefYes) 

Count  134  LogWorth  Difference 

Mean  6.445709  1.0031458  2.45054 

Std  Dev  5.6515639 

T  Hellfire(No) 

Count  122  LogWorth  Difference 

Mean  6.9178279  20.519029  9.1497 

Std  Dev  6.0546644 

'rDAGRRt<11 

Count  82  LogWorth  Difference 

Mean  5.5484756  0.3652872  2.23668 
Std  Dev  5.5960363 

-rDAGRRt>=11 

Count  34 

Mean  12.417647 

Std  Dev  4.8079067 

, - 1 - 1  , - 1 - ,  , - 1 - , 

►  Candidates 

T  HeimreRt<1 1 

Count  91 

Mean  5.6593407 

Std  Dev  5.7494633 

HellfireRt>=1 1 

Count  43 

Mean  8.1098837 

Std  Dev  5.1121096 

►  Candidates 

►  Candidates 

TAPKWSRt<11 

Count  95 

Mean  4.8928947 

Std  Dev  4.9182474 

r  APKWSRt>=11 

Count  27 

Mean  14.042593 

Std  Dev  3.9313264 

►  Candidates 

>  Candidates 

’  NumDAGR>=17 

Count  36 

Mean  4.29375 

Std  Dev  4.3709527 

,rNumDAGR<17 

Count  46 

Mean  6.5304348 

Std  Dev  6.2660481 

►  Candidates 

►  Candidates 
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b.  UA  V  Scenario 

The  results  from  the  UAV  summary  are  quite  similar  to  those  from  the 
MH-60R  summary.  Looking  at  Table  7,  which  shows  the  overall  split  between  each 
missile,  LOGIR  is  once  again  leading  the  way  with  a  mean  of  8.07,  followed  by  DAGR, 
then  APKWS,  then  Hellfire.  Looking  to  Table  8  for  a  more  in-depth  breakdown,  one  can 
see  that  LOGIR  splits  off  first  again,  but  while  LOGIR  with  a  rate  of  1 1  or  greater 
appears  to  lead  the  way,  further  splits  show  that  APKWS  with  a  rate  of  fire  of  1 1  or 
greater  has  a  slightly  higher  mean;  11.97  versus  11.24.  However,  as  stated  in  the 
previous  section,  it  is  more  likely  that  the  LOGIR  will  have  a  greater  rate  of  fire,  with  it 
being  a  fire-and-forget  type  of  missile. 


Table  7.  A  basic  partition  tree  of  the  UAV  summary  data  showing  the  breakdown 

of  the  four  air-to-surface  missile  options  by  the  mean  number  of  Red  casualties 
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Table  8.  From  the  UAV  Summary  data.  A  partition  tree  of  the  four  air-to-surface 

missile  options  and  all  of  their  input  options  based  on  the  mean  number  of  Red 

casualties 


c.  Overall 

The  summary  data  for  both  versions  of  the  scenario  seem  to  agree  that  the 
APWKS  is  the  air-to-surface  missile  of  choice.  This  holds  true  only  as  long  as  the  rate  of 
fire  of  the  APKWS  is  9.1  seconds  or  faster.  If  the  rate  is  slower  than  9.1  seconds,  then 
the  LOGIR  with  a  rate  of  9.1  seconds  or  faster  is  the  best  missile  option.  Since  the 
LOGIR  is  a  fire-and-forget  type  of  missile,  it  is  more  likely  to  have  this  higher  rate  of 
fire. 


3.  Most  Important  Factors  in  Missile  Selection 

The  question  regarding  which  factors  most  impact  the  MOE  of  a  selected  missile 
is  the  same  in  both  versions  of  the  scenario  and  so  data  from  both  output  summaries  will 
be  viewed. 


a.  MH-60R  Scenario 

When  analyzing  mean  total  Red  casualties  as  broken  down  per  missile 

type,  effects  screening  identifies  the  rate  of  fire  of  each  missile  as  the  only  statistically 
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significant,  or  close  to  statistically  significant,  factor.  This  result  is  shown  in  Table  9. 
The  only  exception  to  this  result  is  the  NLOS  system,  in  which  effects  screening 
identifies  both  the  rate  of  fire  and  the  Pk  as  being  statistically  significant  (p  <  .05).  The 
reason  behind  this  potentially  lies  in  the  fact  that  the  NLOS  Pk  was  varied  over  a  much 
broader  range  than  the  other  missiles’  Pks.  Of  note  is  that  when  the  NLOS  data  is  put 
into  a  partition  tree,  it  splits  first  based  on  rate  of  fire,  which  implies  that  rate  of  fire  is 
more  significant  than  the  Pk.  This  is  also  evident  in  the  overall  MH-60R  summary 
partition  tree  in  which  the  rate  of  fire  of  the  NLOS  is  the  second  split  in  the  data,  coming 
only  after  the  split  between  whether  the  LCS  is  equipped  with  NLOS  or  Harpoon.  This 
was  seen  previously  in  Table  3. 


Table  9.  The  parameter  estimates  of  the  regression  analysis  resulting  from  effects 

screening  of  mean  total  Red  casualties  in  the  MH-60R  scenario  separated  by 
missile  type  (Hellfire,  LOGIR,  APKWS,  DAGR,  NLOS,  and  Harpoon, 

respectively) 


Parameter  Estimates 


Term 

Intercept 

NumHellfire 

HellfirePK 

HellfireRt 

HellfireRng 


Estimate 

28.447506 

-0.43597 

-35.47884 

0.349551 

0.0001994 


Std  Error 

17.09077 

0.345671 

22.71844 

0.181176 

0.000793 


Rate  of  fire 
is  most 
statistically 
significant 


Parameter  Estimates 


Term 

Estimate 

Std  Error 

t  Ratio 

Intercept 

-6.878955 

6.997866 

-0.98 

NumLOGIR 

0.0337985 

0.059705 

0.57 

LOGIRPK 

-2.945522 

8.613624 

-0.34 

LOGIRRt 

1.2319363 

0.127247 

9.68 

LOGIRRng 

0.0005192 

0.000959 

0.54 

Prob>|t| 

0.3274 

0.5723 

0.7329 

.0001’ 

,5890. 


Rate  of  fire  is 
most 

statistically 

significant 


Parameter  Estimates 


Term 

Intercept 

NumAPKWS 

APKWSPK 

APKWSRt 

APKWSRng 


Estimate 

-10.79361 

0.0276469 

-20.28933 

1.1960358 

0.0038893 


Std  Error 

13.2737 

0.072218 

12.44339 

0.197015 

0.002514 


Rate  of  Fire  is 
by  far  the  most 
statistically 
significant 
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Parameter  Estimates  Kate  of  fire  is 


Term 

Estimate 

Std  Error 

t  Ratio 

|frot»|tK 

/most 

Intercept 

-3.166745 

11.34594 

-0.28 

j 0.7807  \ 

/  statistically 

NumDAGR 

0.1460278 

0.122586 

1.19  I 

0.2361  J/ 

significant 

DAGRPK 

-2.108077 

15.24337 

-0.14  1 

l  0.8903  / 

DAGRRt 

0.8391069 

0.216147 

3  88 

\0.0002*y 

DAGRRng 

0.0003805 

0.001069 

0.36 

\j22]/ 

w  Parameter  Estimates  ^ 

Term  Estimate  Std  Error  t  Ratio /Prob>|t| 

Intercept  -7.557176  1.121909  -6.74  f  <0001  *-4  statistically 

NLOSPK  12.940663  1.774032  7.29  \  <0001*  j  significant 

NLOSRt  1.0539823  0.062665  16.82  V.0001*/ 


Fk  and  rate  of 
fire  are 


Parameter  Estimates 

Rate  of  fire  is 

Term 

Estimate 

Std  Error 

t  Ratio  /Prob>|t\ 

j  statistically 

Intercept 

-3.097265 

5.085149 

-0.61  [  0.5430  V 

significant 

HarpoonPK 

6.478016 

5.764441 

1.12  1  0.2621 j 

HarpoonRt 

0.3524386 

0.110232 

CO 

k) 

0 

0 

O 

4 

b.  UA  V  Scenario 

When  analyzing  mean  Red  casualties  as  broken  down  per  missile  type, 
effects  screening  identifies  the  rate  of  fire  of  the  missile  as  the  most  common  statistically 
significant  factor.  These  effects  screenings  can  be  seen  in  Table  10.  In  the  LOGIR, 
APKWS,  DAGR,  and  Harpoon  analyses,  the  rate  of  fire  is  the  only  statistically 
significant  factor.  The  NLOS  data  shows  both  the  rate  of  fire  and  the  Pk  as  both  being 
statistically  significant,  but  once  again,  this  is  most  likely  due  to  the  large  range  of  the  Pk 
that  was  used  for  the  NLOS.  Similar  to  the  MH-60  data,  when  the  NLOS  data  is  put  into 
a  partition  tree,  the  first  split  is  on  the  rate  of  fire,  identifying  it  as  more  significant  than 
the  Pk.  The  only  oddity  is  the  Hellfire  missile,  in  which  effects  screening  identifies  no 
statistically  significant  factors. 
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Table  10.  The  parameter  estimates  of  the  regression  analysis  resulting  from  effects 
screening  of  mean  total  Red  casualties  in  the  UAV  scenario  separated  by  missile 
type  (Hellfire,  LOGIR,  APKWS,  DAGR,  NLOS,  and  Harpoon  respectively) 


Parameter  Estimates 

_  Nothing  is 

Term 

Estimate 

Std  Error 

t  Ratio  , 

/Prot»|t|\  /  statistically 

Intercept 

23.961973 

17.34564 

A. 38/ 

0.1695  y  significant 

NumHellfire 

0.3221869 

0.611094 

0.53 

0.5989  A 

HellfirePK 

-33.57364 

23.35904 

-1.44 

0.1531  /  i 

HellfireRt 

0.0870608 

0.187037 

0.47\ 

0.6424  J 

HellfireRng 

0.0004026 

0.000817 

0.49 

\^227/ 

Parameter  Estimates 

,  Rate  of  fire  is 

Term 

Estimate 

Std  Error 

t  Ratio 

/Prob>|t|\  / 

statistically 

Intercept 

-2.097096 

7.427855 

-0.28, 

i  0.7781  X 

significant 

NumLOGIR 

0.2100166 

0.124793 

1.68 

0.0947  /] 

LOGIRPK 

-5.373745 

9.139264 

-0.591 

,  0.5575  j  f 

LOGIRRt 

0.9505791 

0.135083 

7.04 

y  <0001*  / 

LOGIRRng 

-0.000157 

0.001017 

-0.15 

V.8776/ 

^  Parameter  Estimates  - .  Kate  of  fire  is 


Term 

Estimate 

Std  Error 

t  Ratio/ 

Prob>|t|  \ 

/  statistically 

Intercept 

-4.452351 

13.50048 

-0.33 

0.7421  \ 

significant 

NumAPKWS 

0.1695402 

0.146024 

1.16 

0.2480  A 

APKWSPK 

-24.27671 

12.7033 

-1.9H 

0.0584  /  J 

APKWSRt 

0.9555766 

0.200609 

4.79, 

<0001*  / 

APKWSRng 

0.0030006 

0.002562 

1.17 ' 

'MT2439A 

Parameter  Estimates 

Rate  of  Fire  is 

Term 

Estimate 

Std  Error 

t  Ratio 

j  statistically 

Intercept 

1.7875214 

12.1466 

oW 

0.8833  \  /  significant 

NumDAGR 

0.1068216 

0.374792 

0.29 

0.7762  V 

DAGRPK 

-0.421228 

15.30179 

-o.ot 

0.9781  n 

DAGRRt 

0.4539336 

0.215983 

2.1 0i 

0.0378’  y 

DAGRRng 

7.6766e-5 

0.001071 

0.07 

MT943Q/ 

Parameter  Estimates 

FK  and  rate  of  fire  are 

Term 

Estimate 

Std  Error 

t  Ratio 

/Prob>|t|\ 

i  statistically  significant 

Intercept 

-8.937351 

1.055838 

-8.46  / 

<.0001*  h 

NLOSPK 

13.919951 

1.669556 

8.341 

<.0001*  J 

NLOSRt 

1.093062 

0.058975 

18.53 

VOOOIl/ 

'r  Parameter  Estimates 


Term 

Estimate 

Std  Error 

t  Ratio 

/Prob>|t| 

Intercept 

-1.321198 

3.419782 

-0.39  / 

0.6995 

HarpoonPK 

3.9027353 

3.876608 

1.01  \ 

0.3149  t 

HarpoonRt 

0.2832501 

0.074132 

3.82 

\n.0002*I 

Rate  of  fire  is 

statistically 

significant 
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c.  Overall 

Analysis  of  both  sets  of  data  shows  that  the  rate  of  fire  is  statistically 
significant  for  the  LOGIR,  APKWS,  DAGR,  NLOS,  and  Harpoon  missile  systems.  This 
supports  the  conclusion  that  the  rate  of  fire  is  the  most  important  factor  in  missile 
selection. 

C.  SUMMARY 

Analysis  of  the  data  points  shows  that  the  NLOS  is  the  preferred  surface-to- 
surface  missile  as  long  as  it  is  able  to  maintain  a  rate  of  fire  of  9. 1  seconds  or  faster.  The 
analysis  of  the  data  on  the  air-to-surface  missiles  shows  that  the  LOGIR  is  superior 
overall,  but  that  the  APKWS  is  slightly  better  if  its  rate  of  fire  is  9.1  seconds  or  faster. 
When  analyzing  how  the  different  factors  contribute  to  the  effectiveness  of  the  missiles,  it 
is  evident  that  the  rate  of  fire  of  the  missiles  is  the  most  important  factor. 
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VI.  CONCLUSIONS  AND  FUTURE  WORK 


A.  CONCLUSIONS 

1.  Which  Surface-to-Surface  Missile  Should  LCS  Deploy  With? 

Based  on  the  data  analysis  the  NLOS-LS  is  superior  to  the  Harpoon  as  long  as  a 
firing  rate  of  9.1  seconds  or  faster  is  maintainable.  The  data  also  shows  that  the  Pk  of  the 
NLOS  does  not  matter  for  the  most  part  as  long  as  the  high  firing  rate  is  maintained. 

2.  What  Type  of  Air-to-Surface  Missile  Should  the  Aircraft  Be  Equipped 
With? 

Based  on  the  analysis  of  the  summary  data,  the  LOGIR  is  best  Air-to-Surface 
missile  option  overall,  especially  if  a  rate  of  fire  of  9.1  seconds  or  faster  can  be 
maintained.  Further  analysis  shows  that  the  APKWS  is  slightly  superior  to  the  LOGIR  if 
its  rate  of  fire  is  9.1  seconds  or  faster,  but  because  the  LOGIR  is  a  fire-and-forget  missile 
that  does  not  require  lasing  of  the  target,  it  is  much  more  likely  that  the  LOGIR  will  be 
able  to  maintain  a  higher  rate  of  fire. 

3.  What  Factor  Is  Most  Important  in  Deciding  Which  Missile  to  Select? 

The  regression  analysis  of  each  missile  and  its  factors  show  that  that  most 
statistically  significant  factor  is  the  rate  of  fire.  The  Pk  of  the  NLOS  is  statistically 
significant  but  that  is  most  likely  due  to  the  broad  range  of  Pk  that  was  used  for  NLOS  in 
the  experiment. 

B.  RECOMMENDATIONS 

The  results  of  this  thesis  support  the  following  recommendations: 

•  The  Navy  should  continue  the  development  of  the  LOGIR  and  APKWS 

•  The  DAGR  is  a  viable  alternative  if  the  costs  rise  for  LOGIR  and  APKWS 

•  The  Navy  should  continue  with  the  development  of  the  NLOS-LS  and 
PAM. 
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•  When  looking  at  future  missile  systems,  the  rate  of  fire  should  be  the  most 
important  deciding  factor,  within  reason.  A  missile  system  that  has  a  high 
rate  of  fire  but  only  four  missiles  would  not  be  very  beneficial  and  neither 
would  a  missile  system  that  has  a  high  rate  of  fire  but  a  very  poor  hit 
probability. 

•  Tactics  should  allow  for  the  LCS  to  use  its  speed  and  maneuverability, 
when  conditions  permit,  to  keep  the  distance  between  it  and  the  enemy 
combatants  beyond  the  enemy’s  detection  and/or  weapons’  range  and  use 
LCS’s  aircraft  and  over-the-horizon  capabilities  to  defeat  the  enemy 
without  exposing  the  LCS  to  direct  hann. 


C.  FUTURE  WORK 

While  working  on  this  thesis  the  following  items  were  identified  as  warranting 
further  research. 

•  Work  the  missile  defense  capability  of  LCS  into  the  simulation. 

•  Include  aircraft  and  subsurface  threats  into  the  simulation. 

•  Rework  the  scenario  in  SimKit  and  compare  the  results. 

•  Conduct  a  cost  benefits  analysis  on  which  options  provide  the  best 
combination  of  combat  effectiveness  and  low  cost. 

•  Investigate  effects  of  communications  failures  between  the  LCS  and  the 
aircraft  on  their  combat  effectiveness. 

•  Rework  the  scenario  to  include  the  frontrunners  of  the  Navy’s  next 
generation  long-range  anti-ship  missile  program. 

•  Look  into  the  implications  of  this  research  and  what  it  might  mean  for 
LCS  tactics  in  the  SUW  environment. 
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•  Test  the  different  LCS  and  aircraft  missile  combinations  in  situations 
involving  different  tactics  on  the  part  of  Blue  and  Red  forces  to  further  test 
the  effectiveness  of  the  different  missile  systems. 

•  Analyze  the  impact  of  using  different  UAVs  in  place  of  the  Fire  Scout. 
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APPENDIX  A.  PERSONALITIES  AND  CAPABILITIES  OF 

SQUADS 


risjL  NAVAL 

zrrcrzS  POSTGRADUATE 
♦  SCHOOL 


Modeling  Summaiy  —  Missile  Boats 


Weapon 

Effective  Range 

Pk 

Rounds 

C-802 

120  km 

0.45 

4 

Sensors  and  Speed:  Basic  surface  search  with  a  detection  range  of  29  km  and 
a  classification  range  of  24  km.  Missile  boats  transit  and  attack  at  a  speed  of 
45  knots. 

Personality  Summary:  The  missile  boats  are  headed  west  at  full  speed  based 
on  intelligence  of  an  LCS  in  the  area.  They  commence  attack  as  soon  as  their 
sensors  are  within  range  to  provide  sufficient  data  to  their  missiles. 


WWW.NPS.EDL' 


™c0?°UATt  Modeling  Summary  —  LCS 


Weapon 

Effective  Range 

Pk 

Rounds 

NLOS 

40  km 

0.16-0.72 

60 

Harpoon 

85  km 

0.8075-0.9405 

8 

Sensors  and  Speed:  Basic  surface  search  with  a  detection  range  of  32  km  and 
a  classification  range  of  26  km.  LCS  transits  and  attacks  at  a  speed  of  35 
knots. 

Personality  Summary:  The  LCS  are  transiting  north  at  35  knots.  They 
commence  attack  as  soon  as  their  sensors  or  the  helicopter/UAV’s  are  within 
range  to  provide  sufficient  data  to  their  missiles. 


WWW.NPS.EDU 
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^  ~UATt  Modeling  Summary  —  MH-60R 


Weapon 

Effective 

Range 

Pk 

Firing  Rate 

Rounds 

Hellfire 

6000 -8000  m 

0.6375-0.7125 

8  —  18  seconds 

4-8 

LOGIR 

4350-5800  m 

0.4225-0.585 

6-16  seconds 

14-38 

APKWS 

4350-5000  m 

0.4875-0.6175 

8-20  seconds 

14-38 

DAGR 

5250 -7000  m 

0.4875-0.6175 

8  —  20  seconds 

8-24 

Sensors  and  Speed;  Basic  surface  search  with  a  detection  range  of  30  km  and 
a  classification  range  of  6.5  km.  The  MH-60  transits  and  attacks  at  a  speed  of 
125  knots. 


rim 


Personality  Summary:  The  MH-60  is  headed  east  to  conduct  surface  search 
based  on  intelligence  of  Enemy  missile  boats  in  the  area.  They  commence 
attack  as  soon  as  their  sensors  are  within  range  to  provide  sufficient  data  to 
their  missiles.  If  out  of  missiles,  the  MH-60  stays  within  sensor  range  to 
provide  targeting  data  for  the  LCS. 

WWW.NPS.EDU 


bbsL  naval 

POSTGRADUATE 
•  SCHOOL 


Modeling  Summary  —  UAV 


Weapon 

Effective 

Range 

Pk 

Firing  Rate 

Rounds 

Hellfire 

6000 -8000  m 

0.6375-0.7125 

8-18  seconds 

2-4 

LOGIR 

4350 -5800  m 

0.4225-0.585 

6-16  seconds 

7-19 

APKWS 

4350 -5000  m 

0.4875-0.6175 

8-20  seconds 

7-19 

DAGR 

5250 -7000  m 

0.4875-0.6175 

8-20  seconds 

8-12 

Sensors  and  Speed:  Basic  surface  search  with  a  detection  range  of  25  km  and 
a  classification  range  of  6  km.  The  UAV  transits  and  attacks  at  a  speed  of  125 
knots. 

Personality  Summary:  The  UAV  is  headed  east  to  conduct  surface  search 
based  on  intelligence  of  Enemy  missile  boats  in  the  area.  They  commence 
attack  as  soon  as  their  sensors  are  within  range  to  provide  sufficient  data  to 
their  missiles.  If  out  of  missiles,  the  UAV  stays  within  sensor  range  to  provide 
targeting  data  for  the  LCS. 

WWW.NPS.EDU 
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APPENDIX  B.  EXPERIMENTAL  DESIGNS 


This  appendix  illustrates  the  Nearly  Orthogonal  Latin  Hypercube  (NOLH)  used  to 
conduct  the  simulation  experiment.  Due  to  the  size  of  the  full  designs,  only  the  first  50 
rows  are  provided. 


A.  MH-60R  SCENARIO  DESIGN 


Ship  S*t*S 

A-t-S  Missile  Missile  #  Hellfires 

#  LOGIR 

APKVS 

ODAGR 

HnfcPk 

LOGIR  Pk 

APKVS  Pk 

OAGRPk 

NLOSPk 

Harpoon 

Pk 

Hellfire 

rate 

LOGIR  fate 

APKVS 

rate 

DAGR 

rate 

Harpoon 
liOSrate  rate 

Hellfire 

MER  (in 

LOGIR 

MER  (in 

APKVS 
MER  (in 

OAGR 

MER  (in 

MH-60R  Pd  LCSPd 

i 

5  4 

14 

1* 

8 

0.6375 

0,4225 

0.4875 

0.4875 

0.16 

0.8075 

5 

6 

5 

5 

4  4 

6000 

4350 

4350 

5250 

0.5 

0.5 

4 

e  s 

38 

38 

24 

0.7125 

0.585 

0.6175 

0.6175 

0.72 

0.8405 

13 

17 

13 

13 

20  10 

8000 

5800 

5000 

7000 

1 

1 

2 

6  4 

It 

37 

8 

0.65 

0.576 

0.592 

0.495 

0.347 

0.895 

8 

16 

8 

II 

18  5 

7491 

4364 

*711 

6229 

0.819 

0.768 

4 

5  8 

26 

23 

21 

0.7 

0.451 

0.53* 

0.517 

0.653 

0.85 

6 

12 

7 

12 

18  6 

683* 

5479 

4(96 

5606 

0.728 

0.87* 

i 

5  8 

23 

35 

20 

0.641 

0.4(1 

0.497 

0.577 

0.48 

0.895 

6 

10 

12 

12 

20  5 

7123 

5176 

4528 

5*38 

0.73 

0.95* 

i 

8  4 

25 

15 

9 

0.709 

0.551 

0.578 

0.519 

0.244 

0.839 

5  10 

12 

12 

13  7 

7609 

5576 

4543 

6596 

0.516 

0.717 

2 

5  4 

33 

38 

II 

0.851 

0.4*3 

0.525 

0.547 

0.69 

0.817 

7 

13 

7 

7 

12  10 

6438 

5372 

4527 

53*6 

0.677 

0,729 

1 

5  4 

38 

21 

12 

0.705 

0.579 

0.556 

0.53 

0.18 

0.838 

9 

10 

12 

7 

5  6 

6145 

5292 

4894 

59*5 

0.818 

0.589 

2 

5  8 

15 

25 

12 

0.647 

0.432 

0.571 

0.584 

0.385 

0.85 

8 

14 

12 

12 

16  7 

7832 

4895 

497? 

5301 

0.816 

0.515 

4 

8  4 

25 

37 

18 

0.659 

0.456 

0.528 

0.538 

0.557 

0.859 

II 

14 

6 

5 

7  10 

6133 

4727 

4715 

58*2 

0.63 

0.679 

4 

6  8 

17 

19 

10 

0.706 

0.553 

0.612 

0.583 

0.507 

0.827 

6 

14 

7 

7 

12  5 

7969 

5034 

4864 

5565 

0.794 

0.625 

t 

8  4 

24 

13 

8 

0.709 

0.569 

0.542 

0.56 

0.453 

0.867 

8 

7 

12 

12 

7  7 

77*2 

*489 

4398 

63*9 

0.761 

0.55 

i 

5  4 

18 

14 

16 

0.638 

0.456 

0.585 

0.59* 

0.466 

0.916 

8 

9 

8 

6 

16  6 

7945 

5397 

4452 

6861 

0.543 

0.977 

2 

8  8 

27 

3* 

21 

0.651 

0.577 

0.584 

0.503 

0.13 

0.889 

11 

10 

12 

12 

II  6 

7550 

5238 

444? 

5562 

0.788 

0.882 

1 

8  8 

24 

23 

22 

0.691 

0.551 

0.598 

0.512 

0.508 

0.937 

11 

13 

12 

12 

6  7 

7139 

4804 

4501 

5860 

0.524 

0.869 

4 

8  5 

36 

22 

16 

0.673 

0.561 

0.506 

0.559 

0.216 

0.888 

II 

10 

12 

12 

9  7 

7844 

5267 

4975 

6685 

0.764 

0.715 

4 

8  4 

33 

31 

10 

0.706 

0.471 

0.54* 
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6 
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15 
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UAV  SCENARIO  DESIGN 


Ship  S-t-S 

A-t-S  Missile  Missile 

tHellfires 

#  LOGIR 

iAPKWS 

#DAGR 

Hellfire  Pk 

LOGIRPk 

APKWSPk 

DAGRPk 

NLOSPk 

Harpoon 

Pk 

Hellfire 

rate 

LOGIR  rate 

APKWS 

rate 

DAGR 

rate 

Harpoon 
NLOSrate  rate 

Hellfire 

MER  (in 

LOGIR 

MER  (in 

APKWS 
MER  (in 

DAGR 

MER  (in 

UAV  Pd 

LCSPd 

1 

5  2  7 

7  8 

0.6375 

0.4225 

0.4875 

0.4875 

0.16 

0.8075 

5 

6 

5 

5 

4 

4 

6000 

4350 
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5250 

0.4 

0.5 

* 

8 

4 

49 

19 

12 

0.7125 

0.585 

0.6175 

0.6175 

0.72 

0.9405 

13 

17 

13 

13 

20 

10 
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I 

1 

2 

8  2  8 

18 

9 

0.65 

0.576 

0.592 

0.495 

0.347 

0.895 

8 

16 

8 

11 

18 

5 
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4364 

4711 
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0.526 

0.768 

4 

5 

3 

13 

11 

10 

0.7 

0.451 

0.534 

0.517 
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0.85 

6 

12 

7 

12 

18 

6 

6834 

5479 

4*96 
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0.495 

0.87* 

1 

5 

2 

11 

17 

10 

0.641 

0.461 
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0.577 

0.48 

0.895 

6 

10 

12 

12 

20 

5 

7123 

5176 

4528 

5438 

0.683 

0.954 

1 

6 

2 

12 

7 

8 

0.709 

0.551 

0.578 

0.519 

0.244 

0.839 

5 

10 

12 

12 

13 

7 

7609 

5576 

4541 

6596 

0.(22 

0.717 

2 

5 

3 

17 

19 

9 

0.651 

0.443 

0.525 

0.547 

0.69 

0.817 

7 

13 

7 

7 

12 

10 

6438 

5372 
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5346 

0.969 

0.729 

1 

5 

3 

19 

11 

12 

0.705 

0.579 
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0.53 

0.18 

0.83! 

9 

10 

12 

7 

5 

6 
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4894 

5945 
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0.589 

2 

5 

3 

18 

12 

10 

0.647 

0.432 

0.571 

0.584 
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0.85 

8 

14 

12 

12 

16 

7 

7832 
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4 

8 

3 

13 

19 

8 
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0.456 
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0.51! 
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11 

14 

6 
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7 

10 
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8  3  9 

10 

12 

0.706 
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0.827 

8 

14 

7 

7 

12 

5 
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5034 

4864 

5565 

0.426 
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1 

8 

2 

12 

10 

11 

0.709 

0.569 

0.542 
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12 

12 

7 

7 
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0.55 

1 

5  3  8 

7 

8 
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0.585 
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6 

6 

16 

6 

7945 
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4452 
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0.977 

2 

6 

2 

14 

17 

9 

0.651 

0.577 

0.584 
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0.33 

0.889 
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10 

12 

12 

11 

6 
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0.697 
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1 

8 

2 

12 

12 
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0.508 

0.937 

13 

13 
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7 
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0.869 

* 

8 

3 
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11 

10 
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0.888 
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10 

12 

12 

9 

7  7644 
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4975 
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2 

17 
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9 
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9 

14 
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1 

6 

2 

18 
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14 
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7 
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2 
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12 
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0.838 
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to 
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7 

6 
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5952 

0.651 

0.726 

* 

5 

2 

11 

15 

9 

0.705 

0.527 

0.565 

0.618 

0.268 

0.92 

7  14 

7 

7 

13 

7 
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4615 

8743 

0.524 

0.977 

3 

8  3  9 

17 

12 

0.689 

0.505 

0.489 

0.587 

0.213 

0.825 

13 

6 

12 

12 

15 

7 

6270 

5496 
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6360 

0.442 

0.984 

4 
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9 

10 

0.663 

0.434 

0.517 

0.576 

0.273 

0.843 

12 

13 

12 

12 

4 

7 
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4425 

5979 

0.962 

0.815 

4 

5 

2 

15 

19 

11 
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0.577 

0.504 

0.534 

0.828 

0.876 

7 

10 

12 

12 

4 

5 
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0.67 

0.95 

3 

5  2  8 

9 

9 

0.655 

0.542 

0.575 
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0.907 

8 

7 

8 

12 

10 

5 
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5432 
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0.983 
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7 
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0.831 
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6 
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4616 
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8 

3 

14 

11 

11 
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0.497 
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5 
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7 
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0.702 

2 
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10 
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13 
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6 

6117 

5142 
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0.58! 

0.48! 

0.476 

0.86! 

8 

9 

8 

12 
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4 

5 

3 

16 
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8 
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0.925 
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14 

12 
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5 

6 
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4991 
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1 

5 

3 

14 

12 
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0.558 
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0.854 
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12 
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7 

6 
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4685 

4669 
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0.788 

1 

8  4  H 
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12 

0.642 

0.549 

0.588 
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0.936 

6 

13 

6 

6 

17 

6 
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(161 
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0.666 

4 
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13 

12 

0.708 
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0.923 
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10 

12 

6 

15 

5 
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4617 
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0.452 

0.877 

2 

8 

2 

10 

17 

12 

0.66 

0.445 

0.499 

0.527 

0.427 

0.813 

9 

17 

12 

6 

6 

5 

7589 

4716 

4836 

5692 

0.953 

0.659 

4 

8  2  7 

8 

12 

0.694 

0.517 

0.565 

0.571 

0.604 

0.86 

13 

6 

12 

6 

18 

5 

6059 

4801 

4472 

5620 

0.877 

0.617 

4 

8 

2 

16 

14 

10 

0.64 

0.562 

0.517 

0.589 

0.517 

0.855 

12 

7 

12 

5 

12 

5 

7718 

5329 

4191 

6250 

0.751 

0.616 

♦ 

8 

2 

18 

15 

12 
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0.613 
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0.894 
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12 

6 

10 

6 

7501 

5567 

4785 
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3 

8 

3 

17 

17 

12 
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0.607 
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8 

8 

6 

6 

12 

6 
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0.6*3 

4 

8 

3 

16 
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0.677 
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II 

12 

6 

18 
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7914 
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5788 
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1 
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13 

12 
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0.583 
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5 

8 

6 
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17 
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5071 

4971 
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2 
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15 

13 

6 

18 

5 

7886 
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* 
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2 

14 

18 
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15 

10 

10 

9 

5 
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3 

18 
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9 

6 

6 

10 

4 
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4 
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18 

8 

0.662 

0.541 
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0.496 

0.66 

0.87 

9 

8 

13 

13 

17 

4 
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6719 
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3 

8 

3 

18 

11 

10 

0.691 
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12 

14 

5 

13 

11 

10 

7875 
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4481 
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1 

5 

2 

12 

14 

12 
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9 

n 
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9 

6 
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1 
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16 

11 
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0.549 

0.596 
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11 

14 
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15 

9 
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4 

8 

2 

18 

9 

9 
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0.436 

0.503 

0.49 

0.46 
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6 

n 

H 

6 

9 

4 
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